-- Logs begin at Mon 2022-08-29 18:15:29 UTC, end at Mon 2022-08-29 19:24:57 UTC. -- Aug 29 18:52:56.161743 dv-cinder-731261-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 18:52:56.171845 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 18:52:56 2022] *** Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: nodename: dv-cinder-731261-6-iscsi Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: machine: x86_64 Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: clock source: unix Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: pcre jit disabled Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: detected number of CPU cores: 4 Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: current working directory: / Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: detected binary path: /usr/bin/uwsgi-core Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: your processes number limit is 47680 Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: your memory page size is 4096 bytes Aug 29 18:52:56.173089 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: detected max file descriptor number: 2048 Aug 29 18:52:56.173941 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: lock engine: pthread robust mutexes Aug 29 18:52:56.173941 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: thunder lock: enabled Aug 29 18:52:56.173941 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 18:52:56.173941 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 18:52:56.212555 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: Python main interpreter initialized at 0x5648b2e759c0 Aug 29 18:52:56.212555 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: python threads support enabled Aug 29 18:52:56.212555 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: your server socket listen backlog is limited to 100 connections Aug 29 18:52:56.212555 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: your mercy for graceful operations on workers is 90 seconds Aug 29 18:52:56.212975 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 18:52:56.212975 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: *** Operational MODE: preforking *** Aug 29 18:52:56.213290 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: *** uWSGI is running in multiple interpreter mode *** Aug 29 18:52:56.213290 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: spawned uWSGI master process (pid: 76719) Aug 29 18:52:56.213501 dv-cinder-731261-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 18:52:56.215134 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: spawned uWSGI worker 1 (pid: 76732, cores: 1) Aug 29 18:52:56.215134 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: spawned uWSGI worker 2 (pid: 76733, cores: 1) Aug 29 18:52:56.215134 dv-cinder-731261-6-iscsi devstack@keystone.service[76719]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 18:53:00.683314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:53:00.683314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: from cryptography.utils import int_from_bytes Aug 29 18:53:00.684132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:53:00.684132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: from cryptography.utils import int_from_bytes Aug 29 18:53:00.684927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:53:00.684927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: from cryptography.utils import int_from_bytes Aug 29 18:53:00.685509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 18:53:00.685509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: from cryptography.utils import int_from_bytes Aug 29 18:53:00.736120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 18:53:00.736120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 18:53:00.736120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] command line args: [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 18:53:00.736480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 18:53:00.736480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ================================================================================ {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 18:53:00.736714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] admin_token = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.736961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] config_dir = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.737147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.737329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] config_source = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.737505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] debug = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.737654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 18:53:00.737741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.737961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 18:53:00.738075 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] command line args: [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 18:53:00.738170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.738268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] insecure_debug = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.738370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 18:53:00.738469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.738570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ================================================================================ {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 18:53:00.738713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.738832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] admin_token = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.738937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] list_limit = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] config_dir = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] log_config_append = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] config_source = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] log_dir = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] debug = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] log_file = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.739997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] insecure_debug = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.740968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] list_limit = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] log_config_append = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] log_dir = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] log_file = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.741964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742634 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] notification_format = cadf {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.742986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.743127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.743294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.743409 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.743531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.743647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.743764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] publish_errors = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.743897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.744937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] standard_threads = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] notification_format = cadf {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] strict_password_check = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] use_eventlog = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] use_journal = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.745980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] use_json = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] publish_errors = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] use_stderr = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] use_syslog = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] watch_log_file = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.746865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.747026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.747160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.747160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.747382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.747493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] standard_threads = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.747608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.747730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] strict_password_check = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.747844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.747955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.748064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.748174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] use_eventlog = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.748384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.748999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] use_journal = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.749176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.749354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] use_json = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.749354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] use_stderr = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.749734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.749734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.750006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] use_syslog = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.750121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.750240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] watch_log_file = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 18:53:00.750352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.750462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.750572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.750691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.750810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.750930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.751925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752799 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.752913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.753949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.754983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.755920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.756931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.757940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.758848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.759981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.760891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.761883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.762957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.763961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.764077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.764077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.764405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.764592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.764777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.764951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.enabled = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.765910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.766990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.767921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.enabled = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.768950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.769976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.770962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.771992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.772904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] auth.external = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] auth.mapped = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773759 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.773991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] auth.password = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] auth.token = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] catalog.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.774943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.775904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] credential.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] credential.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] auth.external = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] auth.mapped = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.776934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] auth.password = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] auth.token = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.777998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] catalog.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.778924 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] credential.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.779880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.780244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] credential.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.780446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.780576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.780697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.780812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.780934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.781965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.782890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.783011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.783143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.783396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.783512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.783668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.783668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.783904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.784847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.785941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.786956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.787886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.788904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.789915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.790955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.password = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.791900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.792933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.793995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.794907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.795021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.795686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.795843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.password = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.796981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.797896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.798984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.799987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800295 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.800880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] policy.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.801922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] receipt.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.802892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.803992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.804245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.804348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.804490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.804690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.804796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] policy.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.804972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.805096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.805222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.805464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.805603 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.805717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] receipt.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.805931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.806056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.806326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.806445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.806559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.806799 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.806914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.807089 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.807212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] revoke.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.807326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.807578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.807696 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.807849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.808036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.808156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.808343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.808466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] role.cache_time = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.808601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.808839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] role.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.809036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] revoke.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.809153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.809265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] role.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.809378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.809549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] role.cache_time = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.809675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] role.list_limit = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.809795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] role.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.810028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] role.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.810144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.810256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] role.list_limit = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.810481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.810608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.810774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.810963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.811804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.811978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.812166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.812377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.812567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.812746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.812933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.813122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.813122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.813486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.813676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.813676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.813997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.814905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.815026 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.815303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.815421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.815558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.815759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.815877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.816037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.816296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.816510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.816629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.816784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.816950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.817077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.817244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.817410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.817529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.817725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.817849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.817991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.818245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.818362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.818476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.818693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.818820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.818934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.cache_time = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.819172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.819293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.819406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.819623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.819753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.provider = fernet {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.819893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.820049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.820169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.820394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.820509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.820631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.820871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.820988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.821101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.821274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.821391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] trust.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.821625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.821743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.821856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.821972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.822992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.823195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.823322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.cache_time = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.823437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.823550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.823664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.823776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.connection = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.823889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.provider = fernet {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824853 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.824967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.825080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.825193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.825428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.825562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.825679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] trust.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.825792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.825906 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.826019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.826131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.826253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.826543 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.826707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.826896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.827116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.827301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.827489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.827618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.827766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.827945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.828064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.connection = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.828266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.828383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.828518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.828761 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.828900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829130 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.829801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=76732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 18:53:00.829897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.830189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.830366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.830611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.830847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.831128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.831354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.831601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.831834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.832072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.832309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 18:53:00.832475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=76733) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 18:53:00.861538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.861773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.861983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.864349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.864685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.865020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.870762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.878596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.901439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.901671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.906122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.906122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.959799 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:00.962062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.011749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.012199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.012289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.012353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.176993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.177192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.177502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.177605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.177740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.177827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.177968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.178063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.178190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.178274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.178414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.178631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.202798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.203012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.203270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.203479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.203569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.203702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.203834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.203966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.204105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.204239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.204369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.204516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.204649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.204815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.215982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.216151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.229275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.229464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.229606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.229688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.229763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.229899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.229985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.230126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.230208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.230341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.230423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.230554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=76733) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.230639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.230769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=76732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 18:53:01.248158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.249038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:53:01.249376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.249533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.249675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:53:01.249796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:53:01.249928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.250035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.250210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 18:53:01.250325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.250453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.250699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.250781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.250962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.251044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.251245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.260564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.260851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.261123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.261225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.261408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.261524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.261801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.262369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.269011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.269634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.271951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.272236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.272521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.272657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.272804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.272947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.273084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.273238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.273342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.273549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.273630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.273836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.273915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.274100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.274242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.274373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.274522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.274650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.274795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.274925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.275228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.275502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.298508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.299924 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.300039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.300296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.301501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.301793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.305036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.305321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.305601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.306515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.306821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.307131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.312637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.312927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.314588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.314887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.317047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.317334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.317626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.317906 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.318175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.318478 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.318754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.318955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.319054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.319296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.319607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.319964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.320254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.322506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.322802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.323114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.336740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.337038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.339148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.339452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.341051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.343561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.344380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.344680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.346715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.349057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.351550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.352215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.354725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.355464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.355780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.356067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.356351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.356638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.356915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.358453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.358791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.359215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.359513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.359808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.360206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.374143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.374445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.374712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.375012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.377327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.377647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.377893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.378179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.381919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.383604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.385088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.385216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.385417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.386780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.388339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.388639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.388740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.389065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.389149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.389409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.389773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.391968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.392189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.392487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.392803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.393074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.401663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.402029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.402469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.402812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.403204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.403654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.404076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.405215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.405521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.405813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.406373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.406660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.406986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.421328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.423311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.424307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.426430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.427237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.428662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.429442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.430267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.430552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.430846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.430999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.432422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.432711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.433011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.438005 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.439521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.440517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.441773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.442547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.442834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.443145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.443425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.444709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.445015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.445308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.445594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.451460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.452094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.452995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.453954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.454283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.454641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.461161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=76733) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.461433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.461715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.461800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=76732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 18:53:01.461994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.462081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.462266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.462349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.462539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.462623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.462819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.462905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.463111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.463211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.463401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.463487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.463670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.463754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.463941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.464034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.464202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.464285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.464474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.464556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.464740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=76733) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.464825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.465046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.465334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=76732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 18:53:01.501432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5648b2e759c0 pid: 76733 (default app) Aug 29 18:53:01.502101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5648b2e759c0 pid: 76732 (default app) Aug 29 18:53:01.505034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f64db3-6a15-4461-9f13-bba53ffb4fac None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:01.505323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f64db3-6a15-4461-9f13-bba53ffb4fac None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:01.505597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f64db3-6a15-4461-9f13-bba53ffb4fac None None] PATH_INFO: `/v3` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:01.509856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1/1] 10.222.0.94 () {50 vars in 791 bytes} [Mon Aug 29 18:53:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:08.401833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17be97-4264-4ebe-8e0e-1cf9ee75dade None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:08.402390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17be97-4264-4ebe-8e0e-1cf9ee75dade None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:08.402473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17be97-4264-4ebe-8e0e-1cf9ee75dade None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:08.406916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1/2] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:08.412814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:08.413422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:08.413422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:08.484196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: INFO dbcounter [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] Registered counter for database keystone Aug 29 18:53:08.503872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG oslo_db.sqlalchemy.engines [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=76733) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 18:53:08.510924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writer thread running {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 18:53:08.694279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG passlib.handlers.bcrypt [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=76733) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 18:53:08.694577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG passlib.handlers.bcrypt [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=76733) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 18:53:08.754462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:08.755981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:08.887990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 2/3] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 29 18:53:09.343252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17be97-4264-4ebe-8e0e-1cf9ee75dade None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:09.412389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: INFO dbcounter [None req-7f17be97-4264-4ebe-8e0e-1cf9ee75dade None None] Registered counter for database keystone Aug 29 18:53:09.433901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG oslo_db.sqlalchemy.engines [None req-7f17be97-4264-4ebe-8e0e-1cf9ee75dade None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=76732) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 18:53:09.440235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writer thread running {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 18:53:09.562626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17be97-4264-4ebe-8e0e-1cf9ee75dade None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:09.566937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccdb6a-0f83-4c82-be50-9394a0958b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:09.567593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccdb6a-0f83-4c82-be50-9394a0958b4d admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:09.567879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccdb6a-0f83-4c82-be50-9394a0958b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:09.568157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccdb6a-0f83-4c82-be50-9394a0958b4d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:09.576464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccdb6a-0f83-4c82-be50-9394a0958b4d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:09.629519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING keystone.common.rbac_enforcer.enforcer [None req-39ccdb6a-0f83-4c82-be50-9394a0958b4d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 18:53:09.629784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccdb6a-0f83-4c82-be50-9394a0958b4d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:09.640595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 2/4] 10.222.0.94 () {58 vars in 1324 bytes} [Mon Aug 29 18:53:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 18:53:09.646556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:09.661301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f87cb5-7d97-438b-afec-1addab0dba3d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:09.665686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b393178-33d0-4774-b1a3-682112996f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:09.666772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393178-33d0-4774-b1a3-682112996f94 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:09.667054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393178-33d0-4774-b1a3-682112996f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:09.667360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393178-33d0-4774-b1a3-682112996f94 admin admin] PATH_INFO: `/v3/projects/11b2e035c0894ef7a5efbac9016e862c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:09.668931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b393178-33d0-4774-b1a3-682112996f94 admin admin] RBAC: Authorizing `identity:get_project(project_id=11b2e035c0894ef7a5efbac9016e862c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:09.729209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0b393178-33d0-4774-b1a3-682112996f94 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 18:53:09.729550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b393178-33d0-4774-b1a3-682112996f94 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:09.735036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 3/5] 10.222.0.94 () {56 vars in 1207 bytes} [Mon Aug 29 18:53:09 2022] GET /identity/v3/projects/11b2e035c0894ef7a5efbac9016e862c => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:11.655862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1279-9448-4308-b944-c58108593938 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:11.656381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1279-9448-4308-b944-c58108593938 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:11.656487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1279-9448-4308-b944-c58108593938 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:11.660517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 3/6] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:11.666132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df595f-0245-4881-be33-8e27b567c62d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:11.666540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df595f-0245-4881-be33-8e27b567c62d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:11.666683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df595f-0245-4881-be33-8e27b567c62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:11.686943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-39df595f-0245-4881-be33-8e27b567c62d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:11.688504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-39df595f-0245-4881-be33-8e27b567c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:11.697073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 4/7] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 18:53:11.970539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c1279-9448-4308-b944-c58108593938 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:11.983314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c1279-9448-4308-b944-c58108593938 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:11.987526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686d03-a08d-4491-be58-765fa9823d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:11.988125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686d03-a08d-4491-be58-765fa9823d05 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:11.988410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686d03-a08d-4491-be58-765fa9823d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:11.988689 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686d03-a08d-4491-be58-765fa9823d05 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:12.000599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4686d03-a08d-4491-be58-765fa9823d05 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:12.002385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4686d03-a08d-4491-be58-765fa9823d05 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:12.011441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 4/8] 10.222.0.94 () {58 vars in 1324 bytes} [Mon Aug 29 18:53:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 18:53:12.017640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df595f-0245-4881-be33-8e27b567c62d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:12.027306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df595f-0245-4881-be33-8e27b567c62d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:12.031645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789b14b-265f-4c89-8209-942bba80e277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:12.032270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b14b-265f-4c89-8209-942bba80e277 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:12.032552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b14b-265f-4c89-8209-942bba80e277 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:12.032824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b14b-265f-4c89-8209-942bba80e277 admin admin] PATH_INFO: `/v3/users/2b6bbf928a744f868eafe13f4b7ff775` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:12.034117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789b14b-265f-4c89-8209-942bba80e277 admin admin] RBAC: Authorizing `identity:get_user(user_id=2b6bbf928a744f868eafe13f4b7ff775)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:12.035974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789b14b-265f-4c89-8209-942bba80e277 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:12.037500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 5/9] 10.222.0.94 () {56 vars in 1198 bytes} [Mon Aug 29 18:53:12 2022] GET /identity/v3/users/2b6bbf928a744f868eafe13f4b7ff775 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:14.046002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1e68-1265-43c8-9724-61a42f66f545 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.046553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1e68-1265-43c8-9724-61a42f66f545 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.046553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1e68-1265-43c8-9724-61a42f66f545 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.047916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 5/10] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:14.053402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d379-e17a-41e4-8914-4f22b7674ef8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.053846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d379-e17a-41e4-8914-4f22b7674ef8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.054080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d379-e17a-41e4-8914-4f22b7674ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.072954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e3d1d379-e17a-41e4-8914-4f22b7674ef8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:14.074161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e3d1d379-e17a-41e4-8914-4f22b7674ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:14.088431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d7c90-f98c-4a4b-b4cb-20d0711f3553 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.088636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d7c90-f98c-4a4b-b4cb-20d0711f3553 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.088912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d7c90-f98c-4a4b-b4cb-20d0711f3553 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.090244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 6/11] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:14.096372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.096653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.096934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.142255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG passlib.handlers.bcrypt [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=76732) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 18:53:14.142694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG passlib.handlers.bcrypt [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=76732) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 18:53:14.168550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 6/12] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:14.193603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:14.195260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:14.301204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 7/13] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:14.449931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d379-e17a-41e4-8914-4f22b7674ef8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:14.482787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:14.567495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d379-e17a-41e4-8914-4f22b7674ef8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:14.576746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:14.610056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:14.610713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.610994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.611298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.629147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:14.638810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:14.640007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:14.640945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 7/14] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:14.645792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:14.699777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a8d76-3ac5-43b4-82e8-27deeaa38360 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:14.702981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:14.733180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:14.739686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:14.740550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.740836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.741113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.754472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a2bb-1d9d-45c0-8a2b-5d371e8d9f4e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:14.792196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:14.792820 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.793102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.793426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.794108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:14.794448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:14.795932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:14.811493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 8/15] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:14.817270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:14.820870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:14.822751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:14.927883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395f04b-7fb0-469c-819f-29fdceeca52f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:14.935836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 8/16] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:53:14.942793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:14.945867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:14.967642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:14.968218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:14.968544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:14.968819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] PATH_INFO: `/v3/users/2b6bbf928a744f868eafe13f4b7ff775` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:14.970137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] RBAC: Authorizing `identity:get_user(user_id=2b6bbf928a744f868eafe13f4b7ff775)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:14.971734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:14.973346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 9/17] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:14 2022] GET /identity/v3/users/2b6bbf928a744f868eafe13f4b7ff775 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:14.982286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:14.997483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:15.059258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03240e88-5d24-42fb-b342-ae5c0d38aa10 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:15.088824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050effed-8c7b-4c5e-8a76-892baad79333 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:15.094768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:15.095666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:15.095953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:15.096227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:15.116549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:15.118056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:15.125617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:15.126715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:15.126998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:15.127387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:15.128893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:15.130353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:15.131989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 10/18] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:15.137943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:53:15.141889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:15.142954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:53:15.143802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:53:15.143802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:53:15.143802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:15.143802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 9/19] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:53:15.148916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:15.213125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:15.239011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:15.263738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beccaa4-5d02-4de3-bb41-be41756e760a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:15.268043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166dc0-148e-47ac-a4f9-f239645e14e1 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:15.298035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d237c86-6038-4dad-a713-1901f558ade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:15.298646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d237c86-6038-4dad-a713-1901f558ade0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:15.298927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d237c86-6038-4dad-a713-1901f558ade0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:15.299238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d237c86-6038-4dad-a713-1901f558ade0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:15.299968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d237c86-6038-4dad-a713-1901f558ade0 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.domain.id=default, user.id=2b6bbf928a744f868eafe13f4b7ff775) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:15.300344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d237c86-6038-4dad-a713-1901f558ade0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:15.301791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d237c86-6038-4dad-a713-1901f558ade0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:15.303305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671828-b644-4aac-8472-d9006176b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:15.304093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671828-b644-4aac-8472-d9006176b5ee None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:15.304378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671828-b644-4aac-8472-d9006176b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:15.304657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671828-b644-4aac-8472-d9006176b5ee None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:15.305311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e671828-b644-4aac-8472-d9006176b5ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:15.306040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e671828-b644-4aac-8472-d9006176b5ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:15.307455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e671828-b644-4aac-8472-d9006176b5ee None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:15.308381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 11/20] 10.222.0.94 () {56 vars in 1322 bytes} [Mon Aug 29 18:53:15 2022] GET /identity/v3/role_assignments?user.id=2b6bbf928a744f868eafe13f4b7ff775&scope.domain.id=default&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 233 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:15.326487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 10/21] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:17.728946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc4b6c-4794-4a32-80f5-0350ab80e5ff None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:17.729445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc4b6c-4794-4a32-80f5-0350ab80e5ff None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:17.729540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc4b6c-4794-4a32-80f5-0350ab80e5ff None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:17.731165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 12/22] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:17.739744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ae69-db82-4f23-90a0-cb68e258c118 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:17.740255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ae69-db82-4f23-90a0-cb68e258c118 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:17.740336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ae69-db82-4f23-90a0-cb68e258c118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:17.764425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-9717ae69-db82-4f23-90a0-cb68e258c118 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:17.765673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9717ae69-db82-4f23-90a0-cb68e258c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:17.873485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 11/23] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:17.900093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496c7d8-22ac-427c-a989-05f1dc49e546 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:17.900407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496c7d8-22ac-427c-a989-05f1dc49e546 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:17.900660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496c7d8-22ac-427c-a989-05f1dc49e546 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:17.902176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 13/24] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:17.908379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7242ce-bb4d-4e4f-953b-60135bcb4115 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:17.908776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7242ce-bb4d-4e4f-953b-60135bcb4115 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:17.909260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7242ce-bb4d-4e4f-953b-60135bcb4115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:17.914262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266d1c6-265d-4bda-be22-b76fa4da4534 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:17.914889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266d1c6-265d-4bda-be22-b76fa4da4534 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:17.915417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266d1c6-265d-4bda-be22-b76fa4da4534 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:17.916965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 14/25] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:17.923164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e686-7d58-41b0-b664-d73afbb097bd None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:17.923796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e686-7d58-41b0-b664-d73afbb097bd None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:17.924260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e686-7d58-41b0-b664-d73afbb097bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:17.940150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-fb7242ce-bb4d-4e4f-953b-60135bcb4115 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:17.941387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fb7242ce-bb4d-4e4f-953b-60135bcb4115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:17.944808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4162e686-7d58-41b0-b664-d73afbb097bd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:17.946189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4162e686-7d58-41b0-b664-d73afbb097bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:18.023304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 15/26] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:18.025060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12163e60-7ad4-401e-a84a-7b2215385fa2 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:18.025556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12163e60-7ad4-401e-a84a-7b2215385fa2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:18.026031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12163e60-7ad4-401e-a84a-7b2215385fa2 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:18.027556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 16/27] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:18.036745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0168-6318-4aa7-882c-a09351c0f826 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:18.037368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:18.038625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0168-6318-4aa7-882c-a09351c0f826 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:18.039490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0168-6318-4aa7-882c-a09351c0f826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:18.072374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ca9e0168-6318-4aa7-882c-a09351c0f826 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:18.073909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ca9e0168-6318-4aa7-882c-a09351c0f826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:18.124470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 12/28] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:18.173749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 17/29] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:18.303700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7242ce-bb4d-4e4f-953b-60135bcb4115 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:18.440522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7242ce-bb4d-4e4f-953b-60135bcb4115 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:18.500403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:18.501142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:18.501428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:18.501704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:18.518517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0168-6318-4aa7-882c-a09351c0f826 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:18.520776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:18.522508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:18.527730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:18.528649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:18.529457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 13/30] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:18.539101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:18.626154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:18.716643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0168-6318-4aa7-882c-a09351c0f826 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:18.717811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5886e-d8f8-4fe7-aad2-23424e179ae6 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:18.766798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:18.767524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:18.767741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:18.768041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:18.768883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:18.770292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:18.772003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:18.773116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:18.773615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:18.774374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:18.775276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:18.775772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:18.778484 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:18.818429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 14/31] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:18.819829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:18.984700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:18.989033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8dd32-5e7a-4f5d-9933-3877d200ca79 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:19.000835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 18/32] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Aug 29 18:53:18 2022] POST /identity/v3/roles => generated 225 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:53:19.000835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:19.045645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:19.046638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.049964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.050783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.079666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:19.082537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:53:19.121336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:19.122533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:53:19.123720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:53:19.123720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:53:19.123720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:19.128318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 15/33] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:19.130502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:19.153124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:19.206757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f991143-bc1a-48f6-bba1-7fff4b9b8711 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:19.255258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21997073-1094-47d1-805e-995f6084d159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:19.255840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21997073-1094-47d1-805e-995f6084d159 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.256129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21997073-1094-47d1-805e-995f6084d159 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.256406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21997073-1094-47d1-805e-995f6084d159 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.257310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21997073-1094-47d1-805e-995f6084d159 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:19.259445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21997073-1094-47d1-805e-995f6084d159 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:19.290728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 19/34] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Aug 29 18:53:18 2022] POST /identity/v3/roles => generated 219 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:19.292614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21997073-1094-47d1-805e-995f6084d159 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:19.317452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f68a0-9bbd-40bc-8f94-b322bbd2788d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:19.365972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879f5b4-77b6-4ef7-a85a-36f67a822fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:19.366701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879f5b4-77b6-4ef7-a85a-36f67a822fc6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.366918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879f5b4-77b6-4ef7-a85a-36f67a822fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.367229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879f5b4-77b6-4ef7-a85a-36f67a822fc6 None admin] PATH_INFO: `/v3/users/2b6bbf928a744f868eafe13f4b7ff775` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.369029 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879f5b4-77b6-4ef7-a85a-36f67a822fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=2b6bbf928a744f868eafe13f4b7ff775)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:19.370604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879f5b4-77b6-4ef7-a85a-36f67a822fc6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:19.373674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 16/35] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:19 2022] GET /identity/v3/users/2b6bbf928a744f868eafe13f4b7ff775 => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:19.375172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753034-e076-45a3-a079-d9fd5d970c74 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.375462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753034-e076-45a3-a079-d9fd5d970c74 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.375737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753034-e076-45a3-a079-d9fd5d970c74 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.377582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 17/36] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:19.379203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6753034-e076-45a3-a079-d9fd5d970c74 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:19.404966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:19.448900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21997073-1094-47d1-805e-995f6084d159 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:19.480024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:19.502690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371323c0-0181-4a4f-8663-cedb5c93c091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:19.503381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371323c0-0181-4a4f-8663-cedb5c93c091 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.505244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371323c0-0181-4a4f-8663-cedb5c93c091 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.505244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371323c0-0181-4a4f-8663-cedb5c93c091 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.506010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371323c0-0181-4a4f-8663-cedb5c93c091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:19.506259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371323c0-0181-4a4f-8663-cedb5c93c091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:19.507733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371323c0-0181-4a4f-8663-cedb5c93c091 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:19.511337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6753034-e076-45a3-a079-d9fd5d970c74 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:19.529094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 20/37] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:19.530566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59512a5c-4153-4ba7-8de8-b0a5e56579ac None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.530950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59512a5c-4153-4ba7-8de8-b0a5e56579ac None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.531900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59512a5c-4153-4ba7-8de8-b0a5e56579ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.552662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-59512a5c-4153-4ba7-8de8-b0a5e56579ac None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:19.554187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-59512a5c-4153-4ba7-8de8-b0a5e56579ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:19.557728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b74e65-d908-4098-93ce-b81530f0294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:19.558373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74e65-d908-4098-93ce-b81530f0294f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.558660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74e65-d908-4098-93ce-b81530f0294f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.558936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74e65-d908-4098-93ce-b81530f0294f None admin] PATH_INFO: `/v3/domains/default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.560687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b74e65-d908-4098-93ce-b81530f0294f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:19.562250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b74e65-d908-4098-93ce-b81530f0294f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:19.563908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 18/38] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:19.565080 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53de76-ef90-401f-8b41-835b3efea0b4 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.565365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53de76-ef90-401f-8b41-835b3efea0b4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.565643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53de76-ef90-401f-8b41-835b3efea0b4 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.566862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 19/39] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:19.567588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53de76-ef90-401f-8b41-835b3efea0b4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:19.635597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 21/40] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:19.636694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59512a5c-4153-4ba7-8de8-b0a5e56579ac None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:19.669691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:19.699304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53de76-ef90-401f-8b41-835b3efea0b4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:19.744257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:19.744877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.745164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.745441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.746097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:19.748209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:19.764571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59512a5c-4153-4ba7-8de8-b0a5e56579ac None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:19.784017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:19.807484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5937710-3fac-4e16-88ba-45e6bfcc11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:19.808158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5937710-3fac-4e16-88ba-45e6bfcc11dc None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.808436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5937710-3fac-4e16-88ba-45e6bfcc11dc None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.808709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5937710-3fac-4e16-88ba-45e6bfcc11dc None admin] PATH_INFO: `/v3/domains/default/users/2b6bbf928a744f868eafe13f4b7ff775/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.812197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5937710-3fac-4e16-88ba-45e6bfcc11dc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2b6bbf928a744f868eafe13f4b7ff775, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:19.813861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5937710-3fac-4e16-88ba-45e6bfcc11dc None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:19.829381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 22/41] 10.222.0.94 () {58 vars in 1361 bytes} [Mon Aug 29 18:53:19 2022] PUT /identity/v3/domains/default/users/2b6bbf928a744f868eafe13f4b7ff775/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:19.830630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bae24-7aaa-48c5-9ae7-44b4d3c19cbe None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:19.831173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bae24-7aaa-48c5-9ae7-44b4d3c19cbe None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:19.831659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bae24-7aaa-48c5-9ae7-44b4d3c19cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:19.912006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 20/42] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:53:19 2022] POST /identity/v3/projects => generated 301 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:19.948283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:19.978695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-af4bae24-7aaa-48c5-9ae7-44b4d3c19cbe None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:19.980191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-af4bae24-7aaa-48c5-9ae7-44b4d3c19cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:20.073148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 23/43] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:20.089062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541fa5-b8d6-4b87-bd9c-192350826707 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:20.126794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:20.127472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:20.127762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:20.128040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:20.128727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:20.130272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:20.144749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:20.150955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 21/44] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Aug 29 18:53:19 2022] POST /identity/v3/roles => generated 223 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:53:20.352816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bae24-7aaa-48c5-9ae7-44b4d3c19cbe None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:20.455487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bae24-7aaa-48c5-9ae7-44b4d3c19cbe None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:20.457935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:20.492210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d205a-2344-46fe-a04e-96d930928608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:20.492851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d205a-2344-46fe-a04e-96d930928608 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:20.493138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d205a-2344-46fe-a04e-96d930928608 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:20.493417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d205a-2344-46fe-a04e-96d930928608 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:20.494540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d205a-2344-46fe-a04e-96d930928608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:20.496023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d205a-2344-46fe-a04e-96d930928608 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:20.497631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 24/45] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:20.503534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:20.604787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67631c83-59dd-4698-ab14-53bc99f3ff3c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:20.638702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf63c2-589a-4c82-b4e0-a2275281c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:20.639583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf63c2-589a-4c82-b4e0-a2275281c347 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:20.639870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf63c2-589a-4c82-b4e0-a2275281c347 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:20.640151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf63c2-589a-4c82-b4e0-a2275281c347 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:20.640831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf63c2-589a-4c82-b4e0-a2275281c347 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:20.642280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf63c2-589a-4c82-b4e0-a2275281c347 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:20.670556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 22/46] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:53:20 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:21.892959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082686-fba0-4b81-9aac-d114c35ae895 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:21.893473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082686-fba0-4b81-9aac-d114c35ae895 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:21.893473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082686-fba0-4b81-9aac-d114c35ae895 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:21.894855 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 25/47] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:21.900370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e8ee1-696a-4c58-9423-a36a789e5f97 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:21.900784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e8ee1-696a-4c58-9423-a36a789e5f97 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:21.900993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e8ee1-696a-4c58-9423-a36a789e5f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:21.920580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bd0e8ee1-696a-4c58-9423-a36a789e5f97 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:21.922105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bd0e8ee1-696a-4c58-9423-a36a789e5f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:21.976508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:21.998708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 23/48] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:22.267303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3082686-fba0-4b81-9aac-d114c35ae895 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:22.372214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3082686-fba0-4b81-9aac-d114c35ae895 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:22.407967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:22.408660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.408941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.409216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.420480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:22.425388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:22.426485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:22.427722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 26/49] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:22.432285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e8ee1-696a-4c58-9423-a36a789e5f97 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:22.533075 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e8ee1-696a-4c58-9423-a36a789e5f97 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:22.567631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554f20d-8503-428c-ac19-162973b86a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:22.568256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554f20d-8503-428c-ac19-162973b86a30 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.568541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554f20d-8503-428c-ac19-162973b86a30 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.568822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554f20d-8503-428c-ac19-162973b86a30 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.569487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554f20d-8503-428c-ac19-162973b86a30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:22.569792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554f20d-8503-428c-ac19-162973b86a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:22.571190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554f20d-8503-428c-ac19-162973b86a30 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:22.579635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 24/50] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:22.585821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:22.618935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:22.648279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bfa39-75d0-4e58-a68c-2ecafbf76d90 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.648565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bfa39-75d0-4e58-a68c-2ecafbf76d90 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.648843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bfa39-75d0-4e58-a68c-2ecafbf76d90 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.650195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 25/51] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:22.655692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9229f-cf94-4764-b78f-90e242194b72 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.656035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9229f-cf94-4764-b78f-90e242194b72 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.656251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9229f-cf94-4764-b78f-90e242194b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.682403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-21a9229f-cf94-4764-b78f-90e242194b72 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:22.683667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-21a9229f-cf94-4764-b78f-90e242194b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:22.693574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73807f-b6b7-4cc2-8e26-d0b9a0ce9b51 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:22.720712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:22.731190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:22.732673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.733448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.733448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] PATH_INFO: `/v3/users/2b6bbf928a744f868eafe13f4b7ff775` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.734754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] RBAC: Authorizing `identity:get_user(user_id=2b6bbf928a744f868eafe13f4b7ff775)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:22.736449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:22.738093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 27/52] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:22 2022] GET /identity/v3/users/2b6bbf928a744f868eafe13f4b7ff775 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:22.744138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:22.853409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:22.858486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ed39-d571-49db-a2c2-06bb20dd93d2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:22.860166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 26/53] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:22.894121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:22.894758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:22.895042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:22.895343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:22.896475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:22.897928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:22.899528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 28/54] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:22.906078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9229f-cf94-4764-b78f-90e242194b72 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:23.043137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9229f-cf94-4764-b78f-90e242194b72 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:23.086421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:23.086978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:23.087294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:23.087596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:23.088337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.domain.id=default, user.id=2b6bbf928a744f868eafe13f4b7ff775) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:23.088698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:23.090178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:23.100167 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 27/55] 10.222.0.94 () {56 vars in 1322 bytes} [Mon Aug 29 18:53:22 2022] GET /identity/v3/role_assignments?user.id=2b6bbf928a744f868eafe13f4b7ff775&scope.domain.id=default&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 530 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:23.141672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:23.255310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd4942-c0f7-48fc-ad05-d55a40408916 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:23.294985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394183a-5c2a-4ad3-abe2-831d045bb327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:23.295652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394183a-5c2a-4ad3-abe2-831d045bb327 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:23.295935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394183a-5c2a-4ad3-abe2-831d045bb327 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:23.296206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394183a-5c2a-4ad3-abe2-831d045bb327 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:23.297316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394183a-5c2a-4ad3-abe2-831d045bb327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:23.298753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394183a-5c2a-4ad3-abe2-831d045bb327 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:23.300635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 29/56] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:23.309892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:23.355525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:23.429633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98191e-e39b-4433-aa3d-c74bc03d8075 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:23.465748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e698c-963e-4d1f-8498-14cb5edc30d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:23.466365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e698c-963e-4d1f-8498-14cb5edc30d5 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:23.466649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e698c-963e-4d1f-8498-14cb5edc30d5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:23.466926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e698c-963e-4d1f-8498-14cb5edc30d5 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:23.467621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e698c-963e-4d1f-8498-14cb5edc30d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:23.469118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e698c-963e-4d1f-8498-14cb5edc30d5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:23.498545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 28/57] 10.222.0.94 () {60 vars in 1140 bytes} [Mon Aug 29 18:53:23 2022] POST /identity/v3/projects => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:25.446691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17addd8-9e54-464e-8c2c-e50a24e9f93a None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:25.447243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17addd8-9e54-464e-8c2c-e50a24e9f93a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:25.447325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17addd8-9e54-464e-8c2c-e50a24e9f93a None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:25.448626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 30/58] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:25.453964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19ceb7-916a-4b84-9765-e3e0b091bded None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:25.454393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19ceb7-916a-4b84-9765-e3e0b091bded None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:25.454527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19ceb7-916a-4b84-9765-e3e0b091bded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:25.472522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0f19ceb7-916a-4b84-9765-e3e0b091bded None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:25.473711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0f19ceb7-916a-4b84-9765-e3e0b091bded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:25.540300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 29/59] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:25.821469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17addd8-9e54-464e-8c2c-e50a24e9f93a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:25.859576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:25.928582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17addd8-9e54-464e-8c2c-e50a24e9f93a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:25.963654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad0eb6-fd92-4543-ac2c-6914c7fa3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:25.964290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad0eb6-fd92-4543-ac2c-6914c7fa3b4c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:25.964568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad0eb6-fd92-4543-ac2c-6914c7fa3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:25.964843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad0eb6-fd92-4543-ac2c-6914c7fa3b4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:25.966040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad0eb6-fd92-4543-ac2c-6914c7fa3b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:25.967601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad0eb6-fd92-4543-ac2c-6914c7fa3b4c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:25.969166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 31/60] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:25.974792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19ceb7-916a-4b84-9765-e3e0b091bded None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:25.994473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:26.077828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19ceb7-916a-4b84-9765-e3e0b091bded None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:26.115932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298b252-bc94-4450-93d0-70c6f3c23a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:26.116552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b252-bc94-4450-93d0-70c6f3c23a1b None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:26.116836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b252-bc94-4450-93d0-70c6f3c23a1b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:26.117114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b252-bc94-4450-93d0-70c6f3c23a1b None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:26.117780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298b252-bc94-4450-93d0-70c6f3c23a1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:26.119272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298b252-bc94-4450-93d0-70c6f3c23a1b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:26.148842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 30/61] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:53:25 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:29.233881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840968b-86b3-4346-a633-14a0f2602a10 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.234356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840968b-86b3-4346-a633-14a0f2602a10 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.234628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840968b-86b3-4346-a633-14a0f2602a10 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.236445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 32/62] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:29.241668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b4dfa-c77a-4cdc-a833-ad7795311dfb None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.242446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b4dfa-c77a-4cdc-a833-ad7795311dfb None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.242874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b4dfa-c77a-4cdc-a833-ad7795311dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.268983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-764b4dfa-c77a-4cdc-a833-ad7795311dfb None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:29.270438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-764b4dfa-c77a-4cdc-a833-ad7795311dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:29.307549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e19a67-e170-44a8-89cd-af63aaba75e7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.308065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e19a67-e170-44a8-89cd-af63aaba75e7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.308489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e19a67-e170-44a8-89cd-af63aaba75e7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.310263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 33/63] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:29.316051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b9f23-1ca2-4adc-a4cb-ce002ca5e9db None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.316517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b9f23-1ca2-4adc-a4cb-ce002ca5e9db None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.316949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b9f23-1ca2-4adc-a4cb-ce002ca5e9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.345541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-672b9f23-1ca2-4adc-a4cb-ce002ca5e9db None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:29.351351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-672b9f23-1ca2-4adc-a4cb-ce002ca5e9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:29.410414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 31/64] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:29.466071 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 34/65] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:29.656903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e2c9b-9080-4158-a861-f39da6473366 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.657611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e2c9b-9080-4158-a861-f39da6473366 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.658106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e2c9b-9080-4158-a861-f39da6473366 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.660099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 32/66] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:29.665206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbe397-12eb-4cbc-b8e4-faaa8f00a562 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.665736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbe397-12eb-4cbc-b8e4-faaa8f00a562 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.666233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbe397-12eb-4cbc-b8e4-faaa8f00a562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.688366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b0bbe397-12eb-4cbc-b8e4-faaa8f00a562 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:29.689929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b0bbe397-12eb-4cbc-b8e4-faaa8f00a562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:29.704085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026dabd-d689-4765-8b55-8b545e7f1a4f None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.704708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026dabd-d689-4765-8b55-8b545e7f1a4f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.705199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026dabd-d689-4765-8b55-8b545e7f1a4f None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 33/67] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:29.712063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66818d6-fcc2-445c-8e2d-9e4922b0195d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:29.712703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66818d6-fcc2-445c-8e2d-9e4922b0195d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:29.713198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66818d6-fcc2-445c-8e2d-9e4922b0195d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:29.722164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:29.744168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c66818d6-fcc2-445c-8e2d-9e4922b0195d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:29.747538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c66818d6-fcc2-445c-8e2d-9e4922b0195d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:29.756962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:29.820278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 35/68] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:29.821205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbe397-12eb-4cbc-b8e4-faaa8f00a562 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:29.862313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 34/69] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:29.888247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66818d6-fcc2-445c-8e2d-9e4922b0195d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.015245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbe397-12eb-4cbc-b8e4-faaa8f00a562 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.021750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66818d6-fcc2-445c-8e2d-9e4922b0195d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.068479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5e9f7-5a03-43c7-90c4-edfe3a82445c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.069117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e9f7-5a03-43c7-90c4-edfe3a82445c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.069397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e9f7-5a03-43c7-90c4-edfe3a82445c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.069671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e9f7-5a03-43c7-90c4-edfe3a82445c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.080115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b1881-85f8-4b3d-bba1-ffc7a2230241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.081159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5e9f7-5a03-43c7-90c4-edfe3a82445c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:30.081553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b1881-85f8-4b3d-bba1-ffc7a2230241 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.082035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b1881-85f8-4b3d-bba1-ffc7a2230241 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.082509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b1881-85f8-4b3d-bba1-ffc7a2230241 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.084737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-65a5e9f7-5a03-43c7-90c4-edfe3a82445c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:30.087432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:30.088323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:30.089483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 36/70] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:30.090897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c636e-db1b-45a3-807e-eb7cad8134ae None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.091198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c636e-db1b-45a3-807e-eb7cad8134ae None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.091487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c636e-db1b-45a3-807e-eb7cad8134ae None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.093930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 37/71] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:30.099129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c636e-db1b-45a3-807e-eb7cad8134ae None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.101316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b1881-85f8-4b3d-bba1-ffc7a2230241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-831b1881-85f8-4b3d-bba1-ffc7a2230241 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:30.108365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:30.109249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:30.110028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:30.117131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 35/72] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:30.117131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492bdb6-0db3-42b6-9bfd-af4eaa96c834 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.117131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492bdb6-0db3-42b6-9bfd-af4eaa96c834 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.117131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492bdb6-0db3-42b6-9bfd-af4eaa96c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.138869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a492bdb6-0db3-42b6-9bfd-af4eaa96c834 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:30.140253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a492bdb6-0db3-42b6-9bfd-af4eaa96c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:30.145688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.225863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 36/73] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:30.226723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492bdb6-0db3-42b6-9bfd-af4eaa96c834 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.246477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c636e-db1b-45a3-807e-eb7cad8134ae None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.285871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.286465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.286742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.287019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.287711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:30.288014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:30.289412 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:30.299547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 38/74] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:30.305107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.344848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492bdb6-0db3-42b6-9bfd-af4eaa96c834 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.375185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.389473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.390274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.390561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.390934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.391549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:30.391860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:30.393261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:30.403657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.407271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 37/75] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:30.408466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.424250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c77f6-229a-48f8-9dc6-fe8f7353cbe4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.465748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.466642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.467096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.467543 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.479447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:30.485791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:30.486819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:30.487629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 39/76] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:30.488243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.525356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dfb51-0704-4402-8430-9387081bcc9d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.565013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.565762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.566046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.566326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.568358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:30.569825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:30.571560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 38/77] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:30.572352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.607445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0e40-fa57-4c70-a3d1-e04dbc58e26c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.634067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.643802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.644448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.644735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.645015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] PATH_INFO: `/v3/users/2b6bbf928a744f868eafe13f4b7ff775` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.646177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] RBAC: Authorizing `identity:get_user(user_id=2b6bbf928a744f868eafe13f4b7ff775)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:30.647698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:30.649311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 40/78] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/users/2b6bbf928a744f868eafe13f4b7ff775 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:30.650094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.666575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.686612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af0c47-a8fe-4e4c-b76c-8dabc1a68202 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.724058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.724698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.724979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.725250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.742245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:30.743775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:30.745443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 39/79] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:30.746220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.774171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716adcc3-fb81-4bb8-9106-18b2f97cffe8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.827952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.828504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.828779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.829055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.829728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:30.830036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:30.831491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:30.848177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 41/80] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:30.848177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.871457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8610013-9f29-4e01-a720-286de9194695 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:30.893754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.922957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:30.924612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:30.924923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:30.925356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:30.939803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:30.944893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:30.945955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:30.946724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 40/81] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:30.947378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:30.953488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:30.980310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db821a4-a99f-42aa-a5e5-c2d95619b23e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.021474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.022093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.022373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.022647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:31.023344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:31.024802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:31.054601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 42/82] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:53:30 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:31.055546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:31.072751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ae2a9-861e-4db3-9d3c-6c59bccf0f12 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.114881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.115394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.115691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.115972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:31.174663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:31.215649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:31.217512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:31.251655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968fb9e-1f6c-4a18-bfcc-f3eb50c9b5f8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.295964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:31.302680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.304082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.304620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.305274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:31.389252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 41/83] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:31.390951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:31.416273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:31.417973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:31.494882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:31.529840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c90d2-aa65-45af-bf6a-cf9e82b9587b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.555916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 43/84] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:31.557106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:31.575530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.576189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.576482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.576755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:31.581099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:31.583483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:31.585643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 42/85] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:31 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:31.586466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:31.595667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:31.675089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02873835-7849-4fad-a47a-ad3cda17ced0 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.706291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6c78-0647-4af2-819f-a9e8bb682843 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.714406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.716114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.716633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.717108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:31.718035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:31.718684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:31.720373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:31.729892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 44/86] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:31.731640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:31.748778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.749540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.749834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.750117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:31.751495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:31.752946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:31.754786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 43/87] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:31 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:31.760416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:31.771478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:31.859175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:31.868797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c6648-dcd2-4ed8-aa18-d1e523968de4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.901684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed8e48-daa3-4f14-b7a7-639c5259eb5c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:31.915903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.916600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.916937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.917223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:31.918816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:31.920788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:31.923287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 45/88] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:31 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:31.924300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:31.950392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:31.951416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:31.951919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:31.952621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.050608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:32.053273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:32.066085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:32.069079 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76213e42-ae4e-4fad-8337-8368d130f99c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:32.116689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:32.117781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:32.118072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:32.118357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.119885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:32.122066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:32.128723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 46/89] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:31 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:32.128723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:32.163879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:32.199041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 44/90] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:32.199041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:32.257308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7d681-ac13-4ba9-b523-c28a1a160e08 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:32.304418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:32.305425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:32.305918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:32.306418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.307449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=2b6bbf928a744f868eafe13f4b7ff775) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:32.307963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:32.309673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:32.315782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 47/91] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:32 2022] GET /identity/v3/role_assignments?user.id=2b6bbf928a744f868eafe13f4b7ff775&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:32.317088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:32.342942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd525-e6ae-46e9-9f37-694500a16607 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:32.358121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:32.422247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:32.425984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:32.426578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:32.427160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.429756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=81c4f0f0b041419391d31004afc75480) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:32.430334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:32.434985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:32.457917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 45/92] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:32 2022] GET /identity/v3/role_assignments?user.id=81c4f0f0b041419391d31004afc75480&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:32.459393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:32.544160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:32.554030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5b386-7b8e-4b65-bd4f-13e266d1f276 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:32.647739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:32.648747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:32.649819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:32.652042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.741390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:32.743268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:32.753352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe437b9-8fa2-4fc6-80e3-00123d36a7f8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:32.864089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 48/93] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:32.865572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:32.870184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:32.879901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:32.880570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:32.880858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:32.881142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:32.888815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:32.890368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:32.892188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 46/94] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:32 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:32.898227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:32.976552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:33.006167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56e8e9-0212-41fb-abcb-88eb803db134 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:33.048484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:33.049758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.050363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.050841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] PATH_INFO: `/v3/projects/712524e8b36445d382dcc0702df741b3` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.060635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=712524e8b36445d382dcc0702df741b3)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:33.062499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:33.065363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 49/95] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:32 2022] GET /identity/v3/projects/712524e8b36445d382dcc0702df741b3 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:33.070897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:33.074989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627c47b-6ead-4d42-a6c3-e26f6b117315 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:33.114868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4dfcc-e12d-4cab-b056-9d0fa4daa489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:33.115428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4dfcc-e12d-4cab-b056-9d0fa4daa489 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.115714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4dfcc-e12d-4cab-b056-9d0fa4daa489 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.115977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4dfcc-e12d-4cab-b056-9d0fa4daa489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.116716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4dfcc-e12d-4cab-b056-9d0fa4daa489 None admin] RBAC: Adding query filter params (role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=81c4f0f0b041419391d31004afc75480) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:33.117031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4dfcc-e12d-4cab-b056-9d0fa4daa489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:33.118469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4dfcc-e12d-4cab-b056-9d0fa4daa489 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:33.127438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 47/96] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:32 2022] GET /identity/v3/role_assignments?user.id=81c4f0f0b041419391d31004afc75480&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=377956ded99f409684f20afaaf7052c8 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:33.196145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:33.257087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496a62-e540-4c48-83b0-4bdb75287ce7 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:33.319837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7122e-b59d-476c-9cf2-3735d5127283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:33.320759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7122e-b59d-476c-9cf2-3735d5127283 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.321190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7122e-b59d-476c-9cf2-3735d5127283 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.321611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7122e-b59d-476c-9cf2-3735d5127283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.322792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7122e-b59d-476c-9cf2-3735d5127283 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=712524e8b36445d382dcc0702df741b3, user.id=81c4f0f0b041419391d31004afc75480) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:33.323336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7122e-b59d-476c-9cf2-3735d5127283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:33.325989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7122e-b59d-476c-9cf2-3735d5127283 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:33.336036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 50/97] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:33 2022] GET /identity/v3/role_assignments?user.id=81c4f0f0b041419391d31004afc75480&scope.project.id=712524e8b36445d382dcc0702df741b3&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:33.513973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0ed46-f804-4f97-8fbc-99d7e9c69928 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.514459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0ed46-f804-4f97-8fbc-99d7e9c69928 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.514531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0ed46-f804-4f97-8fbc-99d7e9c69928 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.516160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 48/98] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:33.522164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa515-429f-43bc-9e50-7f747ee0d379 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.522472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa515-429f-43bc-9e50-7f747ee0d379 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.522735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa515-429f-43bc-9e50-7f747ee0d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.542570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-54cfa515-429f-43bc-9e50-7f747ee0d379 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:33.544447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-54cfa515-429f-43bc-9e50-7f747ee0d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:33.630043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 51/99] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:33.789818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe22e1-b471-462e-acdd-befa3fbcf1f6 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.790124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe22e1-b471-462e-acdd-befa3fbcf1f6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.790313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe22e1-b471-462e-acdd-befa3fbcf1f6 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.796531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 49/100] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:33.802909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325760b-9907-4ec6-a703-4b38ba220057 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:33.803253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325760b-9907-4ec6-a703-4b38ba220057 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:33.803531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325760b-9907-4ec6-a703-4b38ba220057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:33.828970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d325760b-9907-4ec6-a703-4b38ba220057 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:33.830271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d325760b-9907-4ec6-a703-4b38ba220057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:33.915689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:33.932663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 52/101] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:34.131746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe22e1-b471-462e-acdd-befa3fbcf1f6 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:34.286603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe22e1-b471-462e-acdd-befa3fbcf1f6 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:34.290122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:34.335117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:34.335801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:34.336092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:34.336374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:34.355581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:34.361701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:34.362653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:34.363466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 50/102] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:34.370459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325760b-9907-4ec6-a703-4b38ba220057 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:34.446329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:34.585730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325760b-9907-4ec6-a703-4b38ba220057 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:34.683572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:34.685200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:34.686013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:34.686776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:34.689091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:34.689905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:34.693257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:34.721658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 53/103] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:34.729273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b5f8f-4d24-4d6c-8c8c-b61ef7293d43 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:34.732533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:34.817018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:34.817719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:34.818014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:34.818297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:34.820759 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:34.822455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:34.824338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 51/104] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:34.830562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:34.887442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:34.909480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:35.028840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef4f32-e7e3-4d8a-844e-ecfee9b95b7d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:35.071099 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71a21b-3e22-43fe-9841-aa08cb8c84f3 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:35.102710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:35.104023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.104544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.105042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.106078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:35.108004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:35.148414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:35.149126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.149361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.149646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] PATH_INFO: `/v3/users/9c917bbf5b8649d48e2b087ccdc6c8af` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.156728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 52/105] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:53:34 2022] POST /identity/v3/projects => generated 302 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:35.187913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] RBAC: Authorizing `identity:get_user(user_id=9c917bbf5b8649d48e2b087ccdc6c8af)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:35.189437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:35.191289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 54/106] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:34 2022] GET /identity/v3/users/9c917bbf5b8649d48e2b087ccdc6c8af => generated 289 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:35.197562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:35.339487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:35.367270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901c470-8bb8-4794-aefe-432594347992 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:35.424245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae3d52-35bf-41de-a2f3-eb43fd06e446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:35.424887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae3d52-35bf-41de-a2f3-eb43fd06e446 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.425177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae3d52-35bf-41de-a2f3-eb43fd06e446 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.425459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae3d52-35bf-41de-a2f3-eb43fd06e446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.535286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae3d52-35bf-41de-a2f3-eb43fd06e446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:35.536784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae3d52-35bf-41de-a2f3-eb43fd06e446 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:35.650781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:35.674506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 53/107] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:53:35.682212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:35.832746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0082228-20ed-472c-9ea5-35dbb6b1774d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:35.843483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:35.880989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ed5e9-c94a-46e7-86df-f83c1cf4791c None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.881713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ed5e9-c94a-46e7-86df-f83c1cf4791c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.881713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ed5e9-c94a-46e7-86df-f83c1cf4791c None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.883038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 54/108] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:35.889570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd9ed3-e9ab-4179-a3b0-b1abd644deac None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.890328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd9ed3-e9ab-4179-a3b0-b1abd644deac None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.890830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd9ed3-e9ab-4179-a3b0-b1abd644deac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.892268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:35.893274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:35.893740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:35.894187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:35.899098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:35.899098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:35.901652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 55/109] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:35 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:35.910716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:35.928216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f3fd9ed3-e9ab-4179-a3b0-b1abd644deac None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:35.930487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f3fd9ed3-e9ab-4179-a3b0-b1abd644deac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:36.104789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 55/110] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:36.108517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faafda7-92d9-4433-b386-89c2a85bfc12 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.109017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faafda7-92d9-4433-b386-89c2a85bfc12 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.109506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faafda7-92d9-4433-b386-89c2a85bfc12 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.114362 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 56/111] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:36.123022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cd9d8-2356-4a10-b5cb-b2618e31a396 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.123391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cd9d8-2356-4a10-b5cb-b2618e31a396 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.123826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cd9d8-2356-4a10-b5cb-b2618e31a396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.175307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a18cd9d8-2356-4a10-b5cb-b2618e31a396 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:36.176785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a18cd9d8-2356-4a10-b5cb-b2618e31a396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:36.211725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f149-1525-4307-82fd-f80b80b71b2b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:36.267321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f58a0b-59ac-4045-a60c-4a2976874ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:36.267947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f58a0b-59ac-4045-a60c-4a2976874ae4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.268240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f58a0b-59ac-4045-a60c-4a2976874ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.268521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f58a0b-59ac-4045-a60c-4a2976874ae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.269269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f58a0b-59ac-4045-a60c-4a2976874ae4 None admin] RBAC: Adding query filter params (role.id=d5d02044c9984ae2ba7ebf348711f5ba, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=9c917bbf5b8649d48e2b087ccdc6c8af) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:36.269584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f58a0b-59ac-4045-a60c-4a2976874ae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:36.271042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f58a0b-59ac-4045-a60c-4a2976874ae4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:36.283837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 56/112] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:35 2022] GET /identity/v3/role_assignments?user.id=9c917bbf5b8649d48e2b087ccdc6c8af&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=d5d02044c9984ae2ba7ebf348711f5ba => generated 259 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:36.285128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661c686-4a96-4482-9617-3c2c679dcab7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.285417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661c686-4a96-4482-9617-3c2c679dcab7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.285697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661c686-4a96-4482-9617-3c2c679dcab7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.291576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 57/113] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:36.293983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580f4f4-cfd1-429a-aacb-49362424d8fd None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.294288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580f4f4-cfd1-429a-aacb-49362424d8fd None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.294604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580f4f4-cfd1-429a-aacb-49362424d8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.354202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 57/114] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:36.355739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-3580f4f4-cfd1-429a-aacb-49362424d8fd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:36.357693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3580f4f4-cfd1-429a-aacb-49362424d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:36.420335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:36.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 58/115] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:36.547246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60a32-f425-4d96-a8da-02099a773c51 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.548046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60a32-f425-4d96-a8da-02099a773c51 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.548903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60a32-f425-4d96-a8da-02099a773c51 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.551527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 58/116] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:36.559675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868d5b7-1e15-4616-bec5-0a603fba45ae None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.560499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868d5b7-1e15-4616-bec5-0a603fba45ae None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.560996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868d5b7-1e15-4616-bec5-0a603fba45ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.584890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-3868d5b7-1e15-4616-bec5-0a603fba45ae None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:36.586997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c60a32-f425-4d96-a8da-02099a773c51 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:36.588461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3868d5b7-1e15-4616-bec5-0a603fba45ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:36.663862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:36.694965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 59/117] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:36.698420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868d5b7-1e15-4616-bec5-0a603fba45ae None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:36.774742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c60a32-f425-4d96-a8da-02099a773c51 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:36.819555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:36.821085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.821576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.822055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.835397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:36.843047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:36.843927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:36.844686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:36.847294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 59/118] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:53:36.857361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:36.863169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:36.873548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868d5b7-1e15-4616-bec5-0a603fba45ae None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:36.992155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187676b5-863d-43ff-ab51-45911d2fe0df None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:36.992690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:36.992982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:36.993275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:36.993555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:36.996395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:37.022701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:37.028431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:37.029547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:37.030393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 60/119] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:53:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:37.031033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.049291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.050736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.051362 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.055336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.056375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:37.056893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.058539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.073215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 60/120] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.074404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.171475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8144509-7f26-4c99-80bd-4fdc089a52be None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.208381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.208968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.209256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.209535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.217476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0cfde-70fa-4d19-8db0-c8d65c137bad None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.221438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:37.226507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:37.227401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:37.228173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 61/121] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:37.232295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.256247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:37.264027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.264882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.265328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.265793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.266713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:37.267258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.269069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.280053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 61/122] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.281199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.335734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:37.345499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f3c08-d57e-414b-998c-88ca0b762d8a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.393341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.394078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.394512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.394895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] PATH_INFO: `/v3/users/2b6bbf928a744f868eafe13f4b7ff775` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.396699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] RBAC: Authorizing `identity:get_user(user_id=2b6bbf928a744f868eafe13f4b7ff775)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.398257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.399958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 62/123] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/users/2b6bbf928a744f868eafe13f4b7ff775 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.400821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.432134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801d5eb-c9de-48a3-8e23-a49b809364a5 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.472585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.473565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.474059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.474617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.487795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:37.493737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:37.494590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:37.495375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:37.503293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 62/124] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:37.504863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:37.506041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.520974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b60c1-b666-4f8f-ada8-e32ce48eba8a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.558285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.558948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.559260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.559557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.560938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:37.561469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.563647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.573839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 63/125] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.579791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.610524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:37.626345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4c513-3c4e-4d8d-8d5d-49f5693eb42b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.665139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.666147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.666622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.667108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.668553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.670716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.672732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 63/126] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.673755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.695360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715de680-52c8-40f9-889e-4e3a3ea5d991 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.735462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.736343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.736449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.736764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.737983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.739694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.741574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 64/127] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.741972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.763770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:37.796537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537613-c6a5-41e5-b75f-9e666409cba9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.834177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.835239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.835717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.836180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.837086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:37.837800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.839406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.849988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 64/128] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.855492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:37.885162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:37.898405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72d92d-a01f-4a73-a177-97513c289c42 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:37.935960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:37.937304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:37.937595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:37.937879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:37.939797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:37.941739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:37.944045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 65/129] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:37.951702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.016667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f14cee-474b-468d-a056-94204d0a48a3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.056281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:38.066281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.067439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.067876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.068554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.070663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.072991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.075293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371956-f49a-4238-ae2c-4d2a2c055a65 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.081260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 65/130] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:37 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:38.081260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.127774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46268446-735c-4009-8217-f419588dd481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.129139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46268446-735c-4009-8217-f419588dd481 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.129899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46268446-735c-4009-8217-f419588dd481 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.130402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46268446-735c-4009-8217-f419588dd481 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/2b6bbf928a744f868eafe13f4b7ff775/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.133489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46268446-735c-4009-8217-f419588dd481 None admin] RBAC: Authorizing `identity:create_grant(project_id=581b1f65f92b42d9855b49a2423ba339, user_id=2b6bbf928a744f868eafe13f4b7ff775, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.135408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46268446-735c-4009-8217-f419588dd481 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.155993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 66/131] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:37 2022] PUT /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/2b6bbf928a744f868eafe13f4b7ff775/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:38.157974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46268446-735c-4009-8217-f419588dd481 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.192036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:38.216915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b407b4-88bc-498e-958b-f77530698bd4 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.275590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.276847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.277421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.279575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.284912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.286464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46268446-735c-4009-8217-f419588dd481 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.287976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.291712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 66/132] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:38 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:38.299672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.329038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.329782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.329995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.330275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.331624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.333060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.339167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 67/133] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:38 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:38.342038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.371388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:38.459811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1ab86-f07e-4b43-8bfd-a9ea979e0da3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.494076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d8778-09e7-446a-b55d-4fe04cfbbe37 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.506724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de40719-0dfe-4891-afd7-118387ed326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.507429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de40719-0dfe-4891-afd7-118387ed326a None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.507716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de40719-0dfe-4891-afd7-118387ed326a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.508571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de40719-0dfe-4891-afd7-118387ed326a None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/81c4f0f0b041419391d31004afc75480/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.510766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de40719-0dfe-4891-afd7-118387ed326a None admin] RBAC: Authorizing `identity:create_grant(project_id=581b1f65f92b42d9855b49a2423ba339, user_id=81c4f0f0b041419391d31004afc75480, role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.513653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de40719-0dfe-4891-afd7-118387ed326a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.532572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 67/134] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:38 2022] PUT /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/81c4f0f0b041419391d31004afc75480/roles/377956ded99f409684f20afaaf7052c8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:38.533940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e55a0-8788-4a15-84e7-0467bda1abb4 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.534239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e55a0-8788-4a15-84e7-0467bda1abb4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.534564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e55a0-8788-4a15-84e7-0467bda1abb4 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.537210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 68/135] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:38.538346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e55a0-8788-4a15-84e7-0467bda1abb4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.567808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab954-3d56-4627-9835-17d50d5fb053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.570117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab954-3d56-4627-9835-17d50d5fb053 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.570405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab954-3d56-4627-9835-17d50d5fb053 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.571714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab954-3d56-4627-9835-17d50d5fb053 None admin] PATH_INFO: `/v3/projects/712524e8b36445d382dcc0702df741b3` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.574048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab954-3d56-4627-9835-17d50d5fb053 None admin] RBAC: Authorizing `identity:get_project(project_id=712524e8b36445d382dcc0702df741b3)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.578974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab954-3d56-4627-9835-17d50d5fb053 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.579443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:38.584200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 68/136] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:38 2022] GET /identity/v3/projects/712524e8b36445d382dcc0702df741b3 => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:38.585727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacc78b-c64c-41f8-8cda-eeacf4a4f4ef None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.587485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacc78b-c64c-41f8-8cda-eeacf4a4f4ef None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.587803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacc78b-c64c-41f8-8cda-eeacf4a4f4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.626482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cdacc78b-c64c-41f8-8cda-eeacf4a4f4ef None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:38.629109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cdacc78b-c64c-41f8-8cda-eeacf4a4f4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:38.760821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e55a0-8788-4a15-84e7-0467bda1abb4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.762668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 69/137] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:38.763528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacc78b-c64c-41f8-8cda-eeacf4a4f4ef None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:38.780312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:38.822768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27a8c7-d5e1-4bba-95e8-cecbe7bcec0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:38.823767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a8c7-d5e1-4bba-95e8-cecbe7bcec0d None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.824248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a8c7-d5e1-4bba-95e8-cecbe7bcec0d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.825990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a8c7-d5e1-4bba-95e8-cecbe7bcec0d None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/81c4f0f0b041419391d31004afc75480/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.828892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27a8c7-d5e1-4bba-95e8-cecbe7bcec0d None admin] RBAC: Authorizing `identity:create_grant(project_id=581b1f65f92b42d9855b49a2423ba339, user_id=81c4f0f0b041419391d31004afc75480, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:38.830933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27a8c7-d5e1-4bba-95e8-cecbe7bcec0d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:38.860671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 69/138] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:38 2022] PUT /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/81c4f0f0b041419391d31004afc75480/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:38.933427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacc78b-c64c-41f8-8cda-eeacf4a4f4ef None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:38.958184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9add16-59f6-4ec1-afc7-3f21b287c033 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.958499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9add16-59f6-4ec1-afc7-3f21b287c033 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.958751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9add16-59f6-4ec1-afc7-3f21b287c033 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:38.960332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 70/139] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:38.972739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf9d51-f372-486a-887d-92ae0535e2e6 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:38.973032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf9d51-f372-486a-887d-92ae0535e2e6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:38.973296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf9d51-f372-486a-887d-92ae0535e2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:39.000106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c3bf9d51-f372-486a-887d-92ae0535e2e6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:39.002663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c3bf9d51-f372-486a-887d-92ae0535e2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:39.006061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:39.006764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:39.007098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:39.007438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] PATH_INFO: `/v3/projects/712524e8b36445d382dcc0702df741b3/users/81c4f0f0b041419391d31004afc75480/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:39.010451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] RBAC: Authorizing `identity:create_grant(project_id=712524e8b36445d382dcc0702df741b3, user_id=81c4f0f0b041419391d31004afc75480, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:39.012537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:39.034920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 70/140] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:38 2022] PUT /identity/v3/projects/712524e8b36445d382dcc0702df741b3/users/81c4f0f0b041419391d31004afc75480/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:39.131936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 71/141] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:39.310752 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:39.404418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:39.461391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15ba56-09db-41f1-aaaf-b7c013c6e89a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:39.466241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf9d51-f372-486a-887d-92ae0535e2e6 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:39.476690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:39.520928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:39.521582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:39.521880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:39.522142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:39.523949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:39.525398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:39.528235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 71/142] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:39.534592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:39.669890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf9d51-f372-486a-887d-92ae0535e2e6 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:39.733967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900396cf-d8b4-4dbf-b1f3-131fe17d1d03 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:39.743643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:39.744298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:39.744582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:39.744858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:39.765024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:39.774831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:39.775730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:39.776477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 72/143] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:53:39.789310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:39.815617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:39.816289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:39.816538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:39.816811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:39.817577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:39.819129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:39.836988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:39.866679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 72/144] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:53:39 2022] POST /identity/v3/users => generated 283 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:39.893995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:39.954290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61a6fe-f26e-48fe-9dcc-57edb8ba6a2a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:40.007523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:40.008161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:40.008447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:40.008736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:40.009424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] RBAC: Adding query filter params (name=reader) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:40.009727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:40.011410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:40.023783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 73/145] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:40.030910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:40.196864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9822d7-acf3-4b2d-af46-397b0ac3687d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:40.246105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:40.246856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:40.247156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:40.247440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] PATH_INFO: `/v3/users/9c917bbf5b8649d48e2b087ccdc6c8af` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:40.251788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] RBAC: Authorizing `identity:get_user(user_id=9c917bbf5b8649d48e2b087ccdc6c8af)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:40.253286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:40.255139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 73/146] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:40 2022] GET /identity/v3/users/9c917bbf5b8649d48e2b087ccdc6c8af => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:40.260970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:40.404032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4b2bf-a17f-4e16-b53b-f44fe6369871 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:40.442304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:40.453741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261c9b9-70ba-4791-907b-c05155e6809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:40.454633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261c9b9-70ba-4791-907b-c05155e6809f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:40.454917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261c9b9-70ba-4791-907b-c05155e6809f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:40.455227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261c9b9-70ba-4791-907b-c05155e6809f None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:40.456408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261c9b9-70ba-4791-907b-c05155e6809f None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:40.457869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261c9b9-70ba-4791-907b-c05155e6809f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:40.459949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 74/147] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:40 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:40.466497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:40.686871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:40.770198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d81cb-dba7-4176-b3e4-525186ab3e57 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:40.875910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3c80a-cef1-4756-b461-e32f85c229c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:40.876659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3c80a-cef1-4756-b461-e32f85c229c6 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:40.876897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3c80a-cef1-4756-b461-e32f85c229c6 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:40.877179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3c80a-cef1-4756-b461-e32f85c229c6 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/9c917bbf5b8649d48e2b087ccdc6c8af/roles/d5d02044c9984ae2ba7ebf348711f5ba` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:40.879842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3c80a-cef1-4756-b461-e32f85c229c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=581b1f65f92b42d9855b49a2423ba339, user_id=9c917bbf5b8649d48e2b087ccdc6c8af, role_id=d5d02044c9984ae2ba7ebf348711f5ba)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:40.881823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3c80a-cef1-4756-b461-e32f85c229c6 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:40.904804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 74/148] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:40 2022] PUT /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339/users/9c917bbf5b8649d48e2b087ccdc6c8af/roles/d5d02044c9984ae2ba7ebf348711f5ba => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:41.138866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6d0b-b297-442a-a77e-5f50d80dbf07 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:41.139196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6d0b-b297-442a-a77e-5f50d80dbf07 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:41.139489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6d0b-b297-442a-a77e-5f50d80dbf07 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:41.141128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 75/149] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:41.146681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499d314-bc63-4e91-ab6c-f4f48119dc5d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:41.146983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499d314-bc63-4e91-ab6c-f4f48119dc5d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:41.147278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499d314-bc63-4e91-ab6c-f4f48119dc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:41.173363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f499d314-bc63-4e91-ab6c-f4f48119dc5d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:41.174600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f499d314-bc63-4e91-ab6c-f4f48119dc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:41.263022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 75/150] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:41.425133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1052-ce31-4bce-9314-911fc562dacf None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:41.425601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1052-ce31-4bce-9314-911fc562dacf None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:41.425601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1052-ce31-4bce-9314-911fc562dacf None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:41.427009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 76/151] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:41.443654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13430a9e-09b8-4ff8-a388-c1195641981b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:41.443962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13430a9e-09b8-4ff8-a388-c1195641981b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:41.444215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13430a9e-09b8-4ff8-a388-c1195641981b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:41.472098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-13430a9e-09b8-4ff8-a388-c1195641981b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:41.473326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-13430a9e-09b8-4ff8-a388-c1195641981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:41.533984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:41.560048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 76/152] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:41.728962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1052-ce31-4bce-9314-911fc562dacf None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:41.890335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1052-ce31-4bce-9314-911fc562dacf None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:41.928458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:41.929098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:41.929378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:41.929655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:41.945887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:41.951951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:41.952827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:41.953598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 77/153] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:41.958988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13430a9e-09b8-4ff8-a388-c1195641981b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:42.070824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:42.211608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13430a9e-09b8-4ff8-a388-c1195641981b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:42.231492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:42.280226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:42.281341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:42.281702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:42.282170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:42.283459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:42.283988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:42.286428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:42.302434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 77/154] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:42.308426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:42.363890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a8ec-0e68-4e3b-9ff6-429200cac3be None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:42.417615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:42.435971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:42.436665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:42.436891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:42.437163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:42.456971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:42.462589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:42.463472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:42.464442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 78/155] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:53:42.475718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:42.523558 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f392c-3820-441c-9ad6-fe615890e652 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:42.589778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:42.591389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:42.591895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:42.592396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] PATH_INFO: `/v3/users/2b6bbf928a744f868eafe13f4b7ff775` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:42.594631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] RBAC: Authorizing `identity:get_user(user_id=2b6bbf928a744f868eafe13f4b7ff775)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:42.596125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:42.599028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 78/156] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:42 2022] GET /identity/v3/users/2b6bbf928a744f868eafe13f4b7ff775 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:42.603513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:42.666329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb3b96-6aab-416d-bd63-71eb81fcbcb2 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:42.696650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:42.750223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc9e81-84d7-492f-a74c-3cb8e93ed578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:42.751279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc9e81-84d7-492f-a74c-3cb8e93ed578 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:42.751765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc9e81-84d7-492f-a74c-3cb8e93ed578 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:42.752235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc9e81-84d7-492f-a74c-3cb8e93ed578 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:42.753121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc9e81-84d7-492f-a74c-3cb8e93ed578 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:42.753626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc9e81-84d7-492f-a74c-3cb8e93ed578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:42.755317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc9e81-84d7-492f-a74c-3cb8e93ed578 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:42.766396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 79/157] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:53:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:42.773052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292d217-313e-467b-aceb-3c3a4a155f49 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:42.774336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becba918-577d-40e4-bbe0-c625aeb20db7 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:42.774897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becba918-577d-40e4-bbe0-c625aeb20db7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:42.775490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becba918-577d-40e4-bbe0-c625aeb20db7 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:42.776869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 80/158] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:42.778932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402258c1-6455-42f1-91f6-b178432c7419 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:42.787674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402258c1-6455-42f1-91f6-b178432c7419 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:42.788026 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402258c1-6455-42f1-91f6-b178432c7419 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:42.789602 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:42.790795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 81/159] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:42 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:42.792009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402258c1-6455-42f1-91f6-b178432c7419 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:42.825989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ab5ae-d3f7-484e-926e-bac6e10e852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:42.827381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ab5ae-d3f7-484e-926e-bac6e10e852b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:42.827908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ab5ae-d3f7-484e-926e-bac6e10e852b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:42.828379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ab5ae-d3f7-484e-926e-bac6e10e852b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:42.972066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ab5ae-d3f7-484e-926e-bac6e10e852b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:42.973686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ab5ae-d3f7-484e-926e-bac6e10e852b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:43.091781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402258c1-6455-42f1-91f6-b178432c7419 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:43.151775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 79/160] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:43.156760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e31c7a-8cc9-413a-b727-cb6914927592 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.156760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e31c7a-8cc9-413a-b727-cb6914927592 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.156760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e31c7a-8cc9-413a-b727-cb6914927592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.180535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f9f55-f702-4e0b-b881-d811e11bc85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:43.181556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f9f55-f702-4e0b-b881-d811e11bc85a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.182056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f9f55-f702-4e0b-b881-d811e11bc85a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.182527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f9f55-f702-4e0b-b881-d811e11bc85a None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.187843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f9f55-f702-4e0b-b881-d811e11bc85a None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:43.189629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f9f55-f702-4e0b-b881-d811e11bc85a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:43.191417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-93e31c7a-8cc9-413a-b727-cb6914927592 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:43.191821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 82/161] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:42 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:43.192980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-93e31c7a-8cc9-413a-b727-cb6914927592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:43.193610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122661f-2724-45ac-9f1a-7142fa9053de None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.194102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122661f-2724-45ac-9f1a-7142fa9053de None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.194576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122661f-2724-45ac-9f1a-7142fa9053de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.206925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:43.230553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a122661f-2724-45ac-9f1a-7142fa9053de None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:43.232125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a122661f-2724-45ac-9f1a-7142fa9053de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:43.309665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:43.332806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 80/162] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:43.333720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e31c7a-8cc9-413a-b727-cb6914927592 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:43.342748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 83/163] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:43.343805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122661f-2724-45ac-9f1a-7142fa9053de None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:43.450596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122661f-2724-45ac-9f1a-7142fa9053de None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:43.479153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e31c7a-8cc9-413a-b727-cb6914927592 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:43.507961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:43.508521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.508804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.509077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.543867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:43.544537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.544820 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.545095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.546341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:43.547850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:43.549591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 81/164] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:43 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:43.558075 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:43.577249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:43.583965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:43.618441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:43.620037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:43.723354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9cc35-214f-4d72-8fed-7d6b2c619e27 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:43.779958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:43.780482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:43.780768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:43.781046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:43.781796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=2b6bbf928a744f868eafe13f4b7ff775) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:43.782106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:43.783688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:43.807721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 82/165] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:43 2022] GET /identity/v3/role_assignments?user.id=2b6bbf928a744f868eafe13f4b7ff775&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:43.811855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 84/166] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:43.818274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:43.954925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:43.993994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1b39c-6c8f-460b-b730-4d1dd835e64a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.052481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.053149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.053433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.053771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.054891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:44.056502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:44.058632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 83/167] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:43 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:44.068232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:44.116703 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:44.227635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:44.242179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04a87-daa1-4688-94af-c280b78d1534 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.277401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f31cc-c4c3-4b4c-863a-676aeaf058f7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.289261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.289935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.290226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.290502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.311810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:44.317414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:44.318532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:44.319389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 84/168] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:53:44.327550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:44.335453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:44.354649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.355294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.355622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.355934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.356863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] RBAC: Adding query filter params (role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=81c4f0f0b041419391d31004afc75480) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:44.357181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:44.358729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:44.377673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 85/169] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/role_assignments?user.id=81c4f0f0b041419391d31004afc75480&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=377956ded99f409684f20afaaf7052c8 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:44.378654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:44.556328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a450-c2ac-481f-b250-f7953b44df81 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.567423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda3857-a81f-4607-b531-51dd01bcec1c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.626539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.627201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.627486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.627758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.629050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:44.629513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:44.631104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:44.645667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:44.654302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 86/170] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:44.660442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cbe0a-7e6e-40d8-857f-21eb1c49cee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.660643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:44.661114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cbe0a-7e6e-40d8-857f-21eb1c49cee9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.661399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cbe0a-7e6e-40d8-857f-21eb1c49cee9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.661679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cbe0a-7e6e-40d8-857f-21eb1c49cee9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.684426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cbe0a-7e6e-40d8-857f-21eb1c49cee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-668cbe0a-7e6e-40d8-857f-21eb1c49cee9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:44.690622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:44.691610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:44.692466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:44.694763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 85/171] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:44.697051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2d22-4ed2-4640-b438-0fa4966b6abb None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.697824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2d22-4ed2-4640-b438-0fa4966b6abb None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.698398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f2d22-4ed2-4640-b438-0fa4966b6abb None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.703348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 86/172] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:44.705393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f2d22-4ed2-4640-b438-0fa4966b6abb None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:44.809875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:44.835528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61446e44-5981-4fb9-97a8-a02309ea747e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.882812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f2d22-4ed2-4640-b438-0fa4966b6abb None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:44.900055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6fb1c-edd9-412f-bdea-37c88ef526ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.900712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6fb1c-edd9-412f-bdea-37c88ef526ee None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.900993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6fb1c-edd9-412f-bdea-37c88ef526ee None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.901268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6fb1c-edd9-412f-bdea-37c88ef526ee None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.902503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6fb1c-edd9-412f-bdea-37c88ef526ee None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:44.904147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6fb1c-edd9-412f-bdea-37c88ef526ee None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:44.905961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 87/173] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:44.912682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c9bbb-6e5a-4481-bfae-6189e27c65aa None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.912682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c9bbb-6e5a-4481-bfae-6189e27c65aa None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.912682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c9bbb-6e5a-4481-bfae-6189e27c65aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.938087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-584c9bbb-6e5a-4481-bfae-6189e27c65aa None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:44.939314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-584c9bbb-6e5a-4481-bfae-6189e27c65aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:44.970623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:44.971275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:44.971554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:44.971823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:44.972515 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:44.972988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:44.974389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:44.992522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 87/174] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:44.995725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:45.069404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:45.087303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 88/175] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:45.088247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c9bbb-6e5a-4481-bfae-6189e27c65aa None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:45.189548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef3954-a3ab-47c4-8d34-fe2d2aee9312 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:45.237913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:45.261713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75692138-2b5b-4b24-aa1e-9181cee02f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:45.262344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692138-2b5b-4b24-aa1e-9181cee02f47 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.262620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692138-2b5b-4b24-aa1e-9181cee02f47 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.262890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692138-2b5b-4b24-aa1e-9181cee02f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.290790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c9bbb-6e5a-4481-bfae-6189e27c65aa None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:45.330066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660fe76-144d-43a0-8aed-74d2c5a17367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:45.330772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fe76-144d-43a0-8aed-74d2c5a17367 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.331056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fe76-144d-43a0-8aed-74d2c5a17367 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.331354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fe76-144d-43a0-8aed-74d2c5a17367 None admin] PATH_INFO: `/v3/users/81c4f0f0b041419391d31004afc75480` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.332508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660fe76-144d-43a0-8aed-74d2c5a17367 None admin] RBAC: Authorizing `identity:get_user(user_id=81c4f0f0b041419391d31004afc75480)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:45.333953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660fe76-144d-43a0-8aed-74d2c5a17367 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:45.335591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 89/176] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:45 2022] GET /identity/v3/users/81c4f0f0b041419391d31004afc75480 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:45.336875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b192752-e5b6-4643-9651-488a7fc4af96 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.337160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b192752-e5b6-4643-9651-488a7fc4af96 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.337487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b192752-e5b6-4643-9651-488a7fc4af96 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.338926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 90/177] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:45.345812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b192752-e5b6-4643-9651-488a7fc4af96 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:45.357565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75692138-2b5b-4b24-aa1e-9181cee02f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:45.359063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75692138-2b5b-4b24-aa1e-9181cee02f47 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:45.490784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 88/178] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:45.492579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd38990-3b81-4eb8-9907-27554af6b01c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.493124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd38990-3b81-4eb8-9907-27554af6b01c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.493401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd38990-3b81-4eb8-9907-27554af6b01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.496049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:45.513591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2bd38990-3b81-4eb8-9907-27554af6b01c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:45.514986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2bd38990-3b81-4eb8-9907-27554af6b01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:45.535262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:45.544987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b192752-e5b6-4643-9651-488a7fc4af96 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:45.580390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcb7d3-47f9-4ea1-aa3b-a2b81f089bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:45.581260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcb7d3-47f9-4ea1-aa3b-a2b81f089bac None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.581260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcb7d3-47f9-4ea1-aa3b-a2b81f089bac None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.581527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcb7d3-47f9-4ea1-aa3b-a2b81f089bac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.595238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 89/179] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:45.600124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd38990-3b81-4eb8-9907-27554af6b01c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:45.691436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcb7d3-47f9-4ea1-aa3b-a2b81f089bac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:45.692952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcb7d3-47f9-4ea1-aa3b-a2b81f089bac None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:45.754044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd38990-3b81-4eb8-9907-27554af6b01c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:45.765746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:45.796279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b99a0-fcaa-460e-962f-7e2196aab7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:45.797077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b99a0-fcaa-460e-962f-7e2196aab7bf None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.797368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b99a0-fcaa-460e-962f-7e2196aab7bf None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.797618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b99a0-fcaa-460e-962f-7e2196aab7bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.811569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 91/180] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:45.811777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b99a0-fcaa-460e-962f-7e2196aab7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:45.813067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25dc80-93d7-4075-af04-beef18a25653 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.813432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25dc80-93d7-4075-af04-beef18a25653 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.813783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25dc80-93d7-4075-af04-beef18a25653 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.815348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 92/181] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-281b99a0-fcaa-460e-962f-7e2196aab7bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:45.817470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:45.818359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 90/182] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:45.823807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5659b7-3985-4458-9e17-dc93d9387105 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.825402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5659b7-3985-4458-9e17-dc93d9387105 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.825553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25dc80-93d7-4075-af04-beef18a25653 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:45.825706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5659b7-3985-4458-9e17-dc93d9387105 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.825706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 91/183] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:45.825706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5659b7-3985-4458-9e17-dc93d9387105 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:45.846739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:45.942590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5659b7-3985-4458-9e17-dc93d9387105 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:45.944724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25dc80-93d7-4075-af04-beef18a25653 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:45.980133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102301e5-1824-4518-ab0a-5558dce870d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:45.981495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102301e5-1824-4518-ab0a-5558dce870d9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.981781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102301e5-1824-4518-ab0a-5558dce870d9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.982051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102301e5-1824-4518-ab0a-5558dce870d9 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.983280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102301e5-1824-4518-ab0a-5558dce870d9 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:45.984790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102301e5-1824-4518-ab0a-5558dce870d9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:45.985862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:45.986614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 92/184] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:45 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:45.986719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.986797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.987046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] PATH_INFO: `/v3/projects/712524e8b36445d382dcc0702df741b3` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.987774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d80ed-cebc-4e9a-87e3-2cac3b44f09c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:45.988057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d80ed-cebc-4e9a-87e3-2cac3b44f09c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:45.988247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] RBAC: Authorizing `identity:get_project(project_id=712524e8b36445d382dcc0702df741b3)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:45.988335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d80ed-cebc-4e9a-87e3-2cac3b44f09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:45.989701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:45.991429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 93/185] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:45 2022] GET /identity/v3/projects/712524e8b36445d382dcc0702df741b3 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:45.992814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.009476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-184d80ed-cebc-4e9a-87e3-2cac3b44f09c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:46.010678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-184d80ed-cebc-4e9a-87e3-2cac3b44f09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:46.038506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:46.088681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 93/186] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:46.090402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f3fba-7742-44c2-8d7e-c02c1a9ad7ff None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.090772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f3fba-7742-44c2-8d7e-c02c1a9ad7ff None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.091056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f3fba-7742-44c2-8d7e-c02c1a9ad7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.109335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-900f3fba-7742-44c2-8d7e-c02c1a9ad7ff None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:46.110457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-900f3fba-7742-44c2-8d7e-c02c1a9ad7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:46.118745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:46.126984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2ada4-d517-400b-9f80-21d75c4e1e09 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.168956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.169642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.169928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.170206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.170885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:46.171226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:46.172609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:46.181480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 94/187] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:46.182204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.197569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 94/188] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:46.204421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f3fba-7742-44c2-8d7e-c02c1a9ad7ff None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.296140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:46.307546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f666d31-e891-4fc3-ba23-70b072becc96 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.349696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.349696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.349696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.349696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.365225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:46.370810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:46.372738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:46.373598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 95/189] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:46.374241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.376090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f3fba-7742-44c2-8d7e-c02c1a9ad7ff None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.416645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.417506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.417785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.418882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.419607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=81c4f0f0b041419391d31004afc75480) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:46.419913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:46.421377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:46.439110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 95/190] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/role_assignments?user.id=81c4f0f0b041419391d31004afc75480&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:46.439110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.454353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:46.520374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0ad4-26e9-4682-9372-7a24be4c9a57 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.566682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.567349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.567644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.567912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.568764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=712524e8b36445d382dcc0702df741b3, user.id=81c4f0f0b041419391d31004afc75480) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:46.569086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:46.570568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:46.585710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 96/191] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/role_assignments?user.id=81c4f0f0b041419391d31004afc75480&scope.project.id=712524e8b36445d382dcc0702df741b3&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:46.585710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.586449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfc5ff-ab04-4219-a045-294d33bdf833 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.618535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:46.645846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.646221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.646510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.646783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] PATH_INFO: `/v3/users/371032f9b0dd48e7b50ee99c84c2822f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.672398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] RBAC: Authorizing `identity:get_user(user_id=371032f9b0dd48e7b50ee99c84c2822f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:46.674170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:46.676317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 96/192] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/users/371032f9b0dd48e7b50ee99c84c2822f => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:46.677500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.755373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07bdf4-8282-4a02-93a3-20852a273812 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.806923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:46.812315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.812315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.812315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.812315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.830788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:46.836507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:46.837428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:46.838210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:46.838979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 97/193] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:46.840496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.862014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae401f01-f821-4dd1-b2a8-7c99fc1c7a4d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.898395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.898932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.899244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.899520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.900197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:46.900498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:46.901877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:46.911184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 97/194] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:46.912037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:46.937576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:46.959007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae062a-595d-4c0c-9319-7edb338425bb None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:46.997158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:46.997795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:46.998070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:46.998352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:46.999691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.000979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.002585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 98/195] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:47.008015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:47.029738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431dbb7-181b-4d47-b2dc-5790963588d1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.060116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:47.073475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.074496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.074879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.075230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.138330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be663e8-2944-4c45-b3ff-3d53f80b9b7b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.155614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.157041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.182459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.183114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.183409 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.183704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.184842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:47.185696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.187374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.201402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 99/196] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:53:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:47.202265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:47.218498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:47.269381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 98/197] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:47.270467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:47.301349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:47.328009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edc39f-1f00-4a17-a5d9-2d0d20f99f33 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.362810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.363473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.363752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.364035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] PATH_INFO: `/v3/users/9c917bbf5b8649d48e2b087ccdc6c8af` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.365608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] RBAC: Authorizing `identity:get_user(user_id=9c917bbf5b8649d48e2b087ccdc6c8af)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.367052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.368703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 100/198] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:47 2022] GET /identity/v3/users/9c917bbf5b8649d48e2b087ccdc6c8af => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:47.372541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:47.377758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b746490-027f-4666-88b9-02693d9c19bb None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.414480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.415013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.415321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.415593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.420635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.420635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.444136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 99/199] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:53:47 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:47.444989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:47.477531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:47.514069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce20b8b-1708-46db-a249-4d2b413ea496 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.552202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:47.560308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.560549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.560835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.561151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] PATH_INFO: `/v3/users/371032f9b0dd48e7b50ee99c84c2822f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.562792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] RBAC: Authorizing `identity:get_user(user_id=371032f9b0dd48e7b50ee99c84c2822f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.564346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.566062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 101/200] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:47 2022] GET /identity/v3/users/371032f9b0dd48e7b50ee99c84c2822f => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:47.566896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:47.586352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc05fed-a7ec-4a12-ae51-b90e297c1a6e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.646918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.647740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.648096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.648441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.649671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.651256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.653647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 100/201] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:47 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:47.654505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:47.702723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e0b55-2391-4222-95b9-a7f1aa9ad65f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.747020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.747744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.748031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.748335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.775421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:47.804358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ca0d-2ccc-4bed-ab73-c11586a4f8e5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:47.853655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.855147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:47.855257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:47.856470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:47.856753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:47.857027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:47.882058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:47.940824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:47.942494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:48.007261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 102/202] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:48.008707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:48.078857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 101/203] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:48.079787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:48.089138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:48.157208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:48.184093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7076-9cec-4b02-a9e3-21560f2e5622 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:48.226542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69448c39-3cf6-4787-bf64-1b913173fed2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:48.238778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:48.240213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:48.241094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:48.241573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:48.242785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=371032f9b0dd48e7b50ee99c84c2822f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:48.243824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:48.245778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:48.263204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 103/204] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:48 2022] GET /identity/v3/role_assignments?user.id=371032f9b0dd48e7b50ee99c84c2822f&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:48.263204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:48.292580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:48.293916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:48.294355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:48.294639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:48.296163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:48.297669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:48.301692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 102/205] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:48 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:48.309770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:48.498545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8178fc-f3d1-48db-a1fe-807ac2e9af93 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:48.499800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a43b1-f0ed-433a-b576-275ce672e747 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:48.519275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:48.519555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:48.551296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:48.552251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:48.552695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:48.553152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:48.554476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] RBAC: Adding query filter params (role.id=d5d02044c9984ae2ba7ebf348711f5ba, scope.project.id=581b1f65f92b42d9855b49a2423ba339, user.id=9c917bbf5b8649d48e2b087ccdc6c8af) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:48.554981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:48.557463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:48.577561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 103/206] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:48 2022] GET /identity/v3/role_assignments?user.id=9c917bbf5b8649d48e2b087ccdc6c8af&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=d5d02044c9984ae2ba7ebf348711f5ba => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:48.587419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d386b9-5fca-4a65-8cd6-21e1d5d720c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:48.588086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d386b9-5fca-4a65-8cd6-21e1d5d720c2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:48.588395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d386b9-5fca-4a65-8cd6-21e1d5d720c2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:48.588668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d386b9-5fca-4a65-8cd6-21e1d5d720c2 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:48.596196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d386b9-5fca-4a65-8cd6-21e1d5d720c2 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:48.598104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d386b9-5fca-4a65-8cd6-21e1d5d720c2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:48.600223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 104/207] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:48 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:48.607530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:48.768738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0064f-a8da-4253-970f-dbfbdd79a910 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:48.808956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e93a37-359a-4f34-90aa-4f882fe67462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:48.809605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e93a37-359a-4f34-90aa-4f882fe67462 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:48.809866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e93a37-359a-4f34-90aa-4f882fe67462 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:48.810166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e93a37-359a-4f34-90aa-4f882fe67462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:48.811546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e93a37-359a-4f34-90aa-4f882fe67462 None admin] RBAC: Adding query filter params (role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=371032f9b0dd48e7b50ee99c84c2822f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:48.811546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e93a37-359a-4f34-90aa-4f882fe67462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:48.813122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e93a37-359a-4f34-90aa-4f882fe67462 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:48.820052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 104/208] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:48 2022] GET /identity/v3/role_assignments?user.id=371032f9b0dd48e7b50ee99c84c2822f&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=377956ded99f409684f20afaaf7052c8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:49.806807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759907-4c8c-467b-b688-207cc4cde131 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:49.807381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759907-4c8c-467b-b688-207cc4cde131 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:49.807381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6759907-4c8c-467b-b688-207cc4cde131 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:49.809040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 105/209] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:49.815816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914e655-4435-4955-a47e-d2ece2bc0ced None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:49.816325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914e655-4435-4955-a47e-d2ece2bc0ced None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:49.816725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914e655-4435-4955-a47e-d2ece2bc0ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:49.838880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5238f4-9bdc-462c-b810-a39ae3eefe3b None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:49.839197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5238f4-9bdc-462c-b810-a39ae3eefe3b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:49.839458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5238f4-9bdc-462c-b810-a39ae3eefe3b None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:49.841978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 106/210] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:49.845664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4914e655-4435-4955-a47e-d2ece2bc0ced None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:49.847683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90cbfc-4730-479f-8287-4b7aea56a913 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:49.847969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90cbfc-4730-479f-8287-4b7aea56a913 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:49.848245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90cbfc-4730-479f-8287-4b7aea56a913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:49.848772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4914e655-4435-4955-a47e-d2ece2bc0ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:49.868612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0d90cbfc-4730-479f-8287-4b7aea56a913 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:49.869825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0d90cbfc-4730-479f-8287-4b7aea56a913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:49.922266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:49.973365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 105/211] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:49.987158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 107/212] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:50.266375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914e655-4435-4955-a47e-d2ece2bc0ced None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:50.347901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90cbfc-4730-479f-8287-4b7aea56a913 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:50.397056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914e655-4435-4955-a47e-d2ece2bc0ced None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:50.437463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:50.438517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:50.439143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:50.439613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:50.452512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:50.458098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:50.459009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:50.459842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:50.462768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 106/213] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:50.469660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:50.475940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90cbfc-4730-479f-8287-4b7aea56a913 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:50.486601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:50.515135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:50.529704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:50.531205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:50.532057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:50.532819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:50.534662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:50.536586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:50.538427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 108/214] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:50.544400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:50.607709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758fb95-1c1a-4198-8d96-1aacc9d31888 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:50.648754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:50.649372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:50.649654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:50.650028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:50.650837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:50.651163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:50.652699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:50.662891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 107/215] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:50.669068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:50.696745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e260008-1d29-48b9-bf27-9f1426bea545 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:50.742836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4d65c-9e40-4013-8582-9d29fa05b910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:50.743831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4d65c-9e40-4013-8582-9d29fa05b910 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:50.744120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4d65c-9e40-4013-8582-9d29fa05b910 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:50.744405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4d65c-9e40-4013-8582-9d29fa05b910 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:50.745087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4d65c-9e40-4013-8582-9d29fa05b910 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:50.746716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4d65c-9e40-4013-8582-9d29fa05b910 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:50.783912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 109/216] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:53:50 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:50.784996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a1e19-e132-4a10-aed8-b4a9add0e98d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:50.785265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a1e19-e132-4a10-aed8-b4a9add0e98d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:50.785482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a1e19-e132-4a10-aed8-b4a9add0e98d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:50.791132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 110/217] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:50.799197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783720a3-a10d-4fb2-84ee-b0f9f98d469b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:50.799197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783720a3-a10d-4fb2-84ee-b0f9f98d469b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:50.799197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783720a3-a10d-4fb2-84ee-b0f9f98d469b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:50.823554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-783720a3-a10d-4fb2-84ee-b0f9f98d469b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:50.827307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-783720a3-a10d-4fb2-84ee-b0f9f98d469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:50.835949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:50.842184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e717fd-9735-4dc4-99d2-59f11233e5af None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:50.874507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:50.894869 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:50.895577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:50.896001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:50.896281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] PATH_INFO: `/v3/users/3d5404b4f65a45f7b5c67c52ad140ef2` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:50.919452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] RBAC: Authorizing `identity:get_user(user_id=3d5404b4f65a45f7b5c67c52ad140ef2)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:50.922701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:50.931108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 108/218] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:50 2022] GET /identity/v3/users/3d5404b4f65a45f7b5c67c52ad140ef2 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:50.936154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:50.941585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 111/219] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:51.004388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39517b3-f5e5-40e0-9af8-c228af27028d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.004674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39517b3-f5e5-40e0-9af8-c228af27028d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.004951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39517b3-f5e5-40e0-9af8-c228af27028d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.006329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 112/220] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:51.012418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3f5a-cea4-495c-8c6c-00692d803b5e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.012704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3f5a-cea4-495c-8c6c-00692d803b5e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.012979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3f5a-cea4-495c-8c6c-00692d803b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.040790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-670b3f5a-cea4-495c-8c6c-00692d803b5e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:51.046133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-670b3f5a-cea4-495c-8c6c-00692d803b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:51.065494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52709d-2c41-4a26-9f04-c96cec040129 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:51.114959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:51.115636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.115918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.116196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.150969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:51.161194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 113/221] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:51.225005 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:51.226566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:51.255265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b3f5a-cea4-495c-8c6c-00692d803b5e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:51.398459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 109/222] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:53:51.421733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:51.435667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:51.452734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b3f5a-cea4-495c-8c6c-00692d803b5e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:51.507450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:51.508165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.508448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.508769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.515795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:51.528140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:51.533179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:51.535243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 114/223] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:53:51.537053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:51.580654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7a8c7-0b3f-4bab-ab90-f3ae99fe555e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:51.622643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:51.623440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.623835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.624110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.625266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:51.626718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:51.628658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 110/224] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:51 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:51.629262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:51.679148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61db0f-4266-46a8-acee-9660893b810d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:51.727845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:51.728495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.729059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.729337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.752481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:51.757971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:51.760211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:51.762762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 115/225] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:53:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:51.767124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:51.779241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87423f91-863b-42b0-b440-e71031b35285 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:51.795111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:51.821116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:51.838911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:51.839882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.840537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.840881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.841665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:51.842036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:51.843787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:51.855715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 111/226] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:51.857195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:51.916517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a7718-63c4-4a7e-a30d-5c342c1d1854 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:51.969616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:51.970878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:51.971362 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:51.972186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:51.973123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=3d5404b4f65a45f7b5c67c52ad140ef2) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:51.973620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:51.975467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:51.983633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 116/227] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:51 2022] GET /identity/v3/role_assignments?user.id=3d5404b4f65a45f7b5c67c52ad140ef2&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:51.984672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:52.080037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab6b93-d8cc-4a09-af2e-4829a03230fb None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:52.088424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:52.114875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06571bf1-72ed-4828-944b-7570f6bc3e83 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:52.155765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:52.156502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:52.156947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:52.157287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:52.158148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:52.158622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:52.160330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:52.160851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:52.161745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:52.162436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:52.162883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] PATH_INFO: `/v3/users/371032f9b0dd48e7b50ee99c84c2822f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:52.164761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=371032f9b0dd48e7b50ee99c84c2822f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:52.166431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:52.170348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 117/228] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:51 2022] GET /identity/v3/users/371032f9b0dd48e7b50ee99c84c2822f => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:52.174519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 112/229] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:53:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:52.180307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:52.185411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:52.296565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:52.358849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec382de-4672-48bf-a5d2-8e7752e77ea3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:52.397267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e2113-5d2b-4ad4-8495-14ae9b87d1da None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:52.403174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:52.404353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:52.404829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:52.405335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:52.406746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:52.409086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:52.411193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 118/230] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:52 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:52.420781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:52.452548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:52.473519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:52.474172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:52.474451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:52.474724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] PATH_INFO: `/v3/users/371032f9b0dd48e7b50ee99c84c2822f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:52.476125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] RBAC: Authorizing `identity:get_user(user_id=371032f9b0dd48e7b50ee99c84c2822f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:52.477650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:52.482607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 113/231] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:52 2022] GET /identity/v3/users/371032f9b0dd48e7b50ee99c84c2822f => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:52.493776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:52.594954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d5758-3ec9-4e6b-816e-ce58d3bc0c41 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:52.635181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:52.637045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:52.637571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:52.637997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f/users/371032f9b0dd48e7b50ee99c84c2822f/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:52.640704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] RBAC: Authorizing `identity:create_grant(project_id=f73f010330ab41eead750a6cbd54038f, user_id=371032f9b0dd48e7b50ee99c84c2822f, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:52.642612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:52.664600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 119/232] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:52 2022] PUT /identity/v3/projects/f73f010330ab41eead750a6cbd54038f/users/371032f9b0dd48e7b50ee99c84c2822f/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:52.763033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3c5f1-9ac8-430e-9a48-a106f69e0491 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:52.788918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:52.842572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd076239-e10d-4509-ac0f-297bfb54a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:52.842572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd076239-e10d-4509-ac0f-297bfb54a9bf None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:52.842572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd076239-e10d-4509-ac0f-297bfb54a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:52.842572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd076239-e10d-4509-ac0f-297bfb54a9bf None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:52.843593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd076239-e10d-4509-ac0f-297bfb54a9bf None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:52.845512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd076239-e10d-4509-ac0f-297bfb54a9bf None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:52.847289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 114/233] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:52 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:52.853989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:52.951133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:52.994772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a9798-54a5-4ff8-933c-5dd9bb08f102 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:53.044852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9296d3-7a25-4fb2-a477-8dc62023ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:53.045892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9296d3-7a25-4fb2-a477-8dc62023ef64 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:53.046356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9296d3-7a25-4fb2-a477-8dc62023ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:53.046779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9296d3-7a25-4fb2-a477-8dc62023ef64 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f/users/371032f9b0dd48e7b50ee99c84c2822f/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:53.049242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9296d3-7a25-4fb2-a477-8dc62023ef64 None admin] RBAC: Authorizing `identity:create_grant(project_id=f73f010330ab41eead750a6cbd54038f, user_id=371032f9b0dd48e7b50ee99c84c2822f, role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:53.050874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9296d3-7a25-4fb2-a477-8dc62023ef64 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:53.069461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 120/234] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:52 2022] PUT /identity/v3/projects/f73f010330ab41eead750a6cbd54038f/users/371032f9b0dd48e7b50ee99c84c2822f/roles/377956ded99f409684f20afaaf7052c8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:53.296849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98c967-8b53-42d0-863a-c7f5aa3e7ad4 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:53.297257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98c967-8b53-42d0-863a-c7f5aa3e7ad4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:53.297660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98c967-8b53-42d0-863a-c7f5aa3e7ad4 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:53.299440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 115/235] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:53.305287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c17b59-b2ae-4ebd-9a03-fb361ff86313 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:53.305593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c17b59-b2ae-4ebd-9a03-fb361ff86313 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:53.305900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c17b59-b2ae-4ebd-9a03-fb361ff86313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:53.331401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-32c17b59-b2ae-4ebd-9a03-fb361ff86313 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:53.332662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-32c17b59-b2ae-4ebd-9a03-fb361ff86313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:53.409092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af4b69-d972-42a8-af48-bc2db54cb928 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:53.409404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af4b69-d972-42a8-af48-bc2db54cb928 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:53.409645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af4b69-d972-42a8-af48-bc2db54cb928 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:53.412476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 116/236] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:53.416974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d99565-c384-4ab1-9b7a-accd7885bb77 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:53.417256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d99565-c384-4ab1-9b7a-accd7885bb77 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:53.417522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d99565-c384-4ab1-9b7a-accd7885bb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:53.418903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 121/237] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:53.440371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-18d99565-c384-4ab1-9b7a-accd7885bb77 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:53.441585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-18d99565-c384-4ab1-9b7a-accd7885bb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:53.542589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 117/238] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:53.805355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c17b59-b2ae-4ebd-9a03-fb361ff86313 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:53.896334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:53.957332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c17b59-b2ae-4ebd-9a03-fb361ff86313 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:53.958637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d99565-c384-4ab1-9b7a-accd7885bb77 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:54.001783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:54.002422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:54.002692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:54.002954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:54.023979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:54.029503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:54.030417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:54.031226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 122/239] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:54.039483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:54.168966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdce3f-a319-4c41-ab21-172edc7c84df None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:54.194558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:54.194877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d99565-c384-4ab1-9b7a-accd7885bb77 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:54.215675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:54.216473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:54.216754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:54.217028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:54.217735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] RBAC: Adding query filter params (name=reader) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:54.218040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:54.219506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:54.233711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:54.235379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 123/240] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:54.241910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:54.274804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:54.275480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:54.275759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:54.276034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:54.277862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:54.279553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:54.281185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 118/241] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:54.288208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:54.463899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc32cc-a642-4ead-bc4b-0f8def60f620 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:54.501090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e10b5-554f-45a9-8f55-1537b4e89034 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:54.516696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:54.517351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:54.517631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:54.517903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] PATH_INFO: `/v3/users/acf2e53755af4e7ea93a6e1e88dd8303` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:54.533920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] RBAC: Authorizing `identity:get_user(user_id=acf2e53755af4e7ea93a6e1e88dd8303)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:54.536650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:54.540092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 124/242] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:54 2022] GET /identity/v3/users/acf2e53755af4e7ea93a6e1e88dd8303 => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:54.547769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:54.577449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285c76-64ae-49ae-980e-f198c758019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:54.578402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285c76-64ae-49ae-980e-f198c758019b None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:54.578680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285c76-64ae-49ae-980e-f198c758019b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:54.578947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285c76-64ae-49ae-980e-f198c758019b None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:54.579650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285c76-64ae-49ae-980e-f198c758019b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:54.581194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285c76-64ae-49ae-980e-f198c758019b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:54.617212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 119/243] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:53:54 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:54.687254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:54.785884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4f996-2c70-4526-bb9a-06cf57ecbe64 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:54.852218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:54.853192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:54.853627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:54.854079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:54.986789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:54.988371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:55.048197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:55.127049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a444810-fff2-482c-818f-d5c9b6802993 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:55.127569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a444810-fff2-482c-818f-d5c9b6802993 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:55.127641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a444810-fff2-482c-818f-d5c9b6802993 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:55.129345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 120/244] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:53:55.137306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b9b6-57f4-4e55-af90-1ec0de5ad45b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:55.137694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b9b6-57f4-4e55-af90-1ec0de5ad45b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:55.137987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b9b6-57f4-4e55-af90-1ec0de5ad45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:55.141696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 125/245] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:55.149952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:55.161727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-3047b9b6-57f4-4e55-af90-1ec0de5ad45b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:55.163172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3047b9b6-57f4-4e55-af90-1ec0de5ad45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:55.218385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:55.280521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 121/246] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:55.288078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f7d5-cd46-4e9d-9d9c-2306b4ada9ab None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:55.289209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1a779-7be3-4ef0-9b85-819ac5cf2b2b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:55.289822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1a779-7be3-4ef0-9b85-819ac5cf2b2b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:55.290292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1a779-7be3-4ef0-9b85-819ac5cf2b2b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:55.292038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 122/247] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:55.302020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e9da4-009c-4de2-9562-0d5577db82ee None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:55.302609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e9da4-009c-4de2-9562-0d5577db82ee None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:55.303095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e9da4-009c-4de2-9562-0d5577db82ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:55.326219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-bf4e9da4-009c-4de2-9562-0d5577db82ee None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:55.327901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bf4e9da4-009c-4de2-9562-0d5577db82ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:55.328061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:55.328833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:55.329206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:55.329554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:55.330800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:55.333149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:55.334480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 126/248] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:55 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:55.345307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:55.361095 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:55.445033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 123/249] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:53:55.466804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4406-5d41-45ec-b377-e5d6ff656ea5 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:55.506562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:55.507220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:55.507491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:55.507765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:55.508502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] RBAC: Adding query filter params (role.id=d5d02044c9984ae2ba7ebf348711f5ba, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=acf2e53755af4e7ea93a6e1e88dd8303) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:55.508809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:55.510268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:55.517694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 127/250] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:55 2022] GET /identity/v3/role_assignments?user.id=acf2e53755af4e7ea93a6e1e88dd8303&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=d5d02044c9984ae2ba7ebf348711f5ba => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:55.772567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e9da4-009c-4de2-9562-0d5577db82ee None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:55.883459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:55.888364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e9da4-009c-4de2-9562-0d5577db82ee None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:55.924891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:55.925542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:55.925823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:55.926095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:55.938366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:55.943514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:55.944461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:55.945241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 124/251] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:55.953872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:56.009217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:56.060084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:56.143499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d973-81c1-4c4e-9f2c-5394f5f73e3a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:56.147821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c4f0-176a-4446-8b21-cef98da57d8a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:56.192506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee04f05-cde7-4f24-a1a7-6858794026e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:56.193164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee04f05-cde7-4f24-a1a7-6858794026e5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.193440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee04f05-cde7-4f24-a1a7-6858794026e5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.193709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee04f05-cde7-4f24-a1a7-6858794026e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.213632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee04f05-cde7-4f24-a1a7-6858794026e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-cee04f05-cde7-4f24-a1a7-6858794026e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:56.219450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:56.220334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:56.221109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 125/252] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:53:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:56.222309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffc9bb-c5d1-4b8c-b21e-c89e4412b1b6 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.222309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffc9bb-c5d1-4b8c-b21e-c89e4412b1b6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.222527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffc9bb-c5d1-4b8c-b21e-c89e4412b1b6 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.224006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 126/253] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:56.228794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffc9bb-c5d1-4b8c-b21e-c89e4412b1b6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:56.248823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a61da3-9e5f-4c33-955a-4f4db8008495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:56.249657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a61da3-9e5f-4c33-955a-4f4db8008495 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.249973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a61da3-9e5f-4c33-955a-4f4db8008495 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.250250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a61da3-9e5f-4c33-955a-4f4db8008495 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.250752 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:56.250987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a61da3-9e5f-4c33-955a-4f4db8008495 None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:56.251323 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a61da3-9e5f-4c33-955a-4f4db8008495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:56.252909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a61da3-9e5f-4c33-955a-4f4db8008495 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:56.274473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 128/254] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:56.285268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9ec9c-ddf1-4893-9e73-b2430de424da None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.285921 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9ec9c-ddf1-4893-9e73-b2430de424da None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.286350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9ec9c-ddf1-4893-9e73-b2430de424da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.345734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1ab9ec9c-ddf1-4893-9e73-b2430de424da None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:56.346890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1ab9ec9c-ddf1-4893-9e73-b2430de424da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:56.382551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffc9bb-c5d1-4b8c-b21e-c89e4412b1b6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:56.453895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:56.454518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.454796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.455064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.455753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:56.456056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:56.457445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:56.477789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 127/255] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:56.478668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:56.496762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 129/256] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:56.497420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9ec9c-ddf1-4893-9e73-b2430de424da None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:56.686445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:56.715538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:56.771901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033ecf5-f50d-4766-b32e-56cc5b2156fa None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:56.794543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9ec9c-ddf1-4893-9e73-b2430de424da None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:56.867815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:56.868463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.869968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.870281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] PATH_INFO: `/v3/users/3d5404b4f65a45f7b5c67c52ad140ef2` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.872442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] RBAC: Authorizing `identity:get_user(user_id=3d5404b4f65a45f7b5c67c52ad140ef2)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:56.874892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:56.879786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 128/257] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:56 2022] GET /identity/v3/users/3d5404b4f65a45f7b5c67c52ad140ef2 => generated 297 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:56.881511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:56.887937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:56.888585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:56.888863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:56.889651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] PATH_INFO: `/v3/users/371032f9b0dd48e7b50ee99c84c2822f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:56.894641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] RBAC: Authorizing `identity:get_user(user_id=371032f9b0dd48e7b50ee99c84c2822f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:56.896131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:56.899143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 130/258] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:56 2022] GET /identity/v3/users/371032f9b0dd48e7b50ee99c84c2822f => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:56.902800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.063182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d409e39-59e8-411f-9a36-3d96cd9690bc None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.115097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85043238-8a38-41c7-b837-1122a60444b8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.117591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:57.118239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:57.118515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:57.118787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:57.119977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:57.126556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:57.128374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 131/259] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:56 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:57.129169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.135936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:57.190208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:57.190915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:57.191139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:57.191412 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:57.202637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:57.208085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:57.208963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:57.209738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 129/260] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:53:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:57.210376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.227094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:57.324754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aad7cc-6d38-4064-baec-d1d1d2eba608 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.349032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d6e49-6d55-4841-b1df-9fdb2535d612 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.363376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:57.364053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:57.364334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:57.364658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f/users/3d5404b4f65a45f7b5c67c52ad140ef2/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:57.366588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f73f010330ab41eead750a6cbd54038f, user_id=3d5404b4f65a45f7b5c67c52ad140ef2, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:57.368066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:57.384717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 130/261] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:53:57 2022] PUT /identity/v3/projects/f73f010330ab41eead750a6cbd54038f/users/3d5404b4f65a45f7b5c67c52ad140ef2/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:53:57.385627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.468648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:57.469288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:57.469575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:57.469846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:57.471712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:57.522207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34527a-85fd-4d61-9488-8e7cb29353a2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.567037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:57.567708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:57.567981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:57.568262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:57.568943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:57.569251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:57.570720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:57.582882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 131/262] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:53:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:57.588733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.623117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:57.624609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:57.626609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:57.744075 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 132/263] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:53:57.751650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.777218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a769337-3b97-47bf-b870-74f9629844dc None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.853774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:57.862418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:57.863087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:57.863420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:57.863693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] PATH_INFO: `/v3/users/371032f9b0dd48e7b50ee99c84c2822f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:57.867758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] RBAC: Authorizing `identity:get_user(user_id=371032f9b0dd48e7b50ee99c84c2822f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:57.869726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:57.871413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 132/264] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:57 2022] GET /identity/v3/users/371032f9b0dd48e7b50ee99c84c2822f => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:57.877466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:57.959400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f714b60-798c-4c4d-b40a-2fb25d0d7f6d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:57.961229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:58.002493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:58.003445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.003727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.004003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.005373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:58.006840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:58.008902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 133/265] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:57 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:58.017088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:58.193134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d6a4e-8939-4d7b-96a0-42e0cf1a1400 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:58.196255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a545de6-83aa-4a30-b29c-4c639754682b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:58.246481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3986b-d6d4-463b-95d0-91f56fcb6723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:58.247168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3986b-d6d4-463b-95d0-91f56fcb6723 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.247426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39c7bd-2e19-4d39-be03-54e596f5bdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:58.247694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3986b-d6d4-463b-95d0-91f56fcb6723 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.247778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3986b-d6d4-463b-95d0-91f56fcb6723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.248159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c7bd-2e19-4d39-be03-54e596f5bdb2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.248441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c7bd-2e19-4d39-be03-54e596f5bdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.248547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3986b-d6d4-463b-95d0-91f56fcb6723 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=371032f9b0dd48e7b50ee99c84c2822f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:58.248713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c7bd-2e19-4d39-be03-54e596f5bdb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.248860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3986b-d6d4-463b-95d0-91f56fcb6723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:58.250300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3986b-d6d4-463b-95d0-91f56fcb6723 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:58.257275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 134/266] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:58 2022] GET /identity/v3/role_assignments?user.id=371032f9b0dd48e7b50ee99c84c2822f&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:58.266616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeea02-f3ce-48a9-b54c-c2a441ce0447 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.266987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeea02-f3ce-48a9-b54c-c2a441ce0447 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.267306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeea02-f3ce-48a9-b54c-c2a441ce0447 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.269303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 135/267] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:58.271176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbf26-067d-4fb3-9141-ac203682443c None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.271461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbf26-067d-4fb3-9141-ac203682443c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.271737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbf26-067d-4fb3-9141-ac203682443c None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.273219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 136/268] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:58.280880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dc642-a1a0-4164-b343-398d9a92554d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.281229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dc642-a1a0-4164-b343-398d9a92554d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.281512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dc642-a1a0-4164-b343-398d9a92554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.321152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-561dc642-a1a0-4164-b343-398d9a92554d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:58.322584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-561dc642-a1a0-4164-b343-398d9a92554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:58.330948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:58.335187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39c7bd-2e19-4d39-be03-54e596f5bdb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:58.336771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39c7bd-2e19-4d39-be03-54e596f5bdb2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:58.363505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:58.411498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 137/269] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 29 18:53:58.419982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc81d-176f-4e18-83c4-76dcbfe66da8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.420463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc81d-176f-4e18-83c4-76dcbfe66da8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.420846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc81d-176f-4e18-83c4-76dcbfe66da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.452512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-60dfc81d-176f-4e18-83c4-76dcbfe66da8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:58.453815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-60dfc81d-176f-4e18-83c4-76dcbfe66da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:58.458760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 133/270] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:53:58.460110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df822587-fe21-4bb6-ba4b-64bff8cad95b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.460398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df822587-fe21-4bb6-ba4b-64bff8cad95b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.460673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df822587-fe21-4bb6-ba4b-64bff8cad95b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.461960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 134/271] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:53:58.468385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df822587-fe21-4bb6-ba4b-64bff8cad95b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:58.556026 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 138/272] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:58.557581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011640f0-313c-4f59-95af-f8f72e4bfddb None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.557935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011640f0-313c-4f59-95af-f8f72e4bfddb None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.558258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011640f0-313c-4f59-95af-f8f72e4bfddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.582204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-011640f0-313c-4f59-95af-f8f72e4bfddb None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:53:58.584258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-011640f0-313c-4f59-95af-f8f72e4bfddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:53:58.649031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:58.658436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df822587-fe21-4bb6-ba4b-64bff8cad95b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:58.703642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 139/273] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:53:58.705011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:58.705716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:58.708322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:58.708623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:58.709986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:58.711470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:58.713664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 135/274] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:53:58 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:58.722573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011640f0-313c-4f59-95af-f8f72e4bfddb None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:58.762038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:58.787433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:58.956850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011640f0-313c-4f59-95af-f8f72e4bfddb None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:58.969972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389c6ad-7a75-408a-859f-f7e5607a6462 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:59.002921 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.003881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.004308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.004705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.005558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] RBAC: Adding query filter params (role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=371032f9b0dd48e7b50ee99c84c2822f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:59.005873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:59.007543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:59.008442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.009137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.009514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.009834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.011006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:59.013476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:59.015084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 140/275] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:53:58 2022] GET /identity/v3/role_assignments?user.id=371032f9b0dd48e7b50ee99c84c2822f&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=377956ded99f409684f20afaaf7052c8 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:59.015275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 136/276] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:53:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:59.015918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.021181 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.068583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:59.171299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb4017-9c88-43ab-b350-8f3ea84ef062 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:59.190545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:59.207379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7bb99-adf2-4ecf-8466-ac5e10e8f206 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:59.230989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.232335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.232883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.233341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.235035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:59.236911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:59.261372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.262032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.262414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.262708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.270922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 137/277] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:53:59 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:53:59.274037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.282185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:59.289019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:59.289987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:53:59.291812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:53:59.296258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 141/278] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:59.306155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.429537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:59.447309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eeee98-994e-4546-be3c-6bc26535abfa None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:59.467188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea68ff-d6de-4171-948d-b20929592ce9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:59.512764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.515168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.515730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.516403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.537158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:53:59.538970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.539684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.539967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.540372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.541214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:59.541524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:53:59.542972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:53:59.544663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:53:59.545528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:53:59.546364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:59.546437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 138/279] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:53:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:53:59.555253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.555976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 142/280] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:59.562477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.580611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:59.717693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342774c6-28e6-4d29-bb6e-a0520f94a56a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:59.746978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513de6af-2473-45e5-995a-cf5caaac4224 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:53:59.777960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.778615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.778884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.779206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.779941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:53:59.780340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:59.781895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:59.797364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:53:59.798021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:53:59.798303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:53:59.799507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] PATH_INFO: `/v3/users/e8982d50fbca413f8a275726631c1131` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:53:59.805440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 139/281] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:53:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:53:59.814788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.816411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] RBAC: Authorizing `identity:get_user(user_id=e8982d50fbca413f8a275726631c1131)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:53:59.817918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:53:59.821363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 143/282] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:59 2022] GET /identity/v3/users/e8982d50fbca413f8a275726631c1131 => generated 293 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:53:59.830387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:53:59.839025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:59.991296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:53:59.994344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430d3c6-be2d-4736-bfb6-05fad518aac4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:00.066302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:00.067697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:00.068211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:00.068744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] PATH_INFO: `/v3/users/acf2e53755af4e7ea93a6e1e88dd8303` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:00.070455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] RBAC: Authorizing `identity:get_user(user_id=acf2e53755af4e7ea93a6e1e88dd8303)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:00.073099 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:00.076599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 140/283] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:53:59 2022] GET /identity/v3/users/acf2e53755af4e7ea93a6e1e88dd8303 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:00.089127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:00.089406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9891c43-b34a-4831-b868-b6eb6ca15d9f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:00.134455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2a36b-1dfe-47b2-8381-348764a893d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:00.135928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2a36b-1dfe-47b2-8381-348764a893d2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:00.136620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2a36b-1dfe-47b2-8381-348764a893d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:00.137940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2a36b-1dfe-47b2-8381-348764a893d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:00.139475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2a36b-1dfe-47b2-8381-348764a893d2 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, user.id=e8982d50fbca413f8a275726631c1131, scope.system=all) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:00.140015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2a36b-1dfe-47b2-8381-348764a893d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:00.142001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2a36b-1dfe-47b2-8381-348764a893d2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:00.148339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 144/284] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Aug 29 18:53:59 2022] GET /identity/v3/role_assignments?user.id=e8982d50fbca413f8a275726631c1131&scope.system=all&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:00.173306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5cfea-12f0-4f31-a746-5cbacf5daace None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:00.173628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5cfea-12f0-4f31-a746-5cbacf5daace None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:00.173858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5cfea-12f0-4f31-a746-5cbacf5daace None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:00.175641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 145/285] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:00.182810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d092a-bf84-4c4c-bcca-8d9155cb788b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:00.183874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d092a-bf84-4c4c-bcca-8d9155cb788b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:00.184374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d092a-bf84-4c4c-bcca-8d9155cb788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:00.206432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:00.231367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0564-8c40-4833-827d-3904badd4edb None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:00.253222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-3b7d092a-bf84-4c4c-bcca-8d9155cb788b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:00.263421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3b7d092a-bf84-4c4c-bcca-8d9155cb788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:00.278541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:00.279660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:00.280151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:00.280709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:00.292105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:00.294804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:00.297165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 141/286] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:00 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:00.305953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:00.406599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 146/287] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:00.447624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b3c4-7f59-4adf-a6b5-868235a38783 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:00.489278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:00.490455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:00.491098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:00.491612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f/users/acf2e53755af4e7ea93a6e1e88dd8303/roles/d5d02044c9984ae2ba7ebf348711f5ba` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:00.493949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] RBAC: Authorizing `identity:create_grant(project_id=f73f010330ab41eead750a6cbd54038f, user_id=acf2e53755af4e7ea93a6e1e88dd8303, role_id=d5d02044c9984ae2ba7ebf348711f5ba)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:00.496574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:00.513147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 142/288] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:54:00 2022] PUT /identity/v3/projects/f73f010330ab41eead750a6cbd54038f/users/acf2e53755af4e7ea93a6e1e88dd8303/roles/d5d02044c9984ae2ba7ebf348711f5ba => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:00.858440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d092a-bf84-4c4c-bcca-8d9155cb788b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:00.982116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:01.038046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d092a-bf84-4c4c-bcca-8d9155cb788b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:01.075492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:01.076454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:01.076750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:01.077002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:01.090445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:01.095983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:01.097034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:01.098373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 147/289] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:01.104695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:01.125188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:01.252362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a7d6-6872-4e36-8cb8-664815c0e981 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:01.289890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:01.290504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:01.290787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:01.291086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:01.291757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:01.292065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:01.293553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:01.302910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 143/290] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:01.308417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:01.462858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae56c35-9e43-4b84-a027-0c45cbd6e6ae None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:01.496915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:01.512752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:01.513663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:01.513950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:01.514231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] PATH_INFO: `/v3/users/3d5404b4f65a45f7b5c67c52ad140ef2` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:01.519936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] RBAC: Authorizing `identity:get_user(user_id=3d5404b4f65a45f7b5c67c52ad140ef2)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:01.521510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:01.523254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 148/291] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:54:01 2022] GET /identity/v3/users/3d5404b4f65a45f7b5c67c52ad140ef2 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:01.530894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:01.636584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:01.666589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fce61-0b1f-4823-93ee-3568323ee1b8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:01.704789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:01.705414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:01.705699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:01.705978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:01.786452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:01.787871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:01.876364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:01.912227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 144/292] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:01.920187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:02.080257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1ff17-8dba-4816-8f47-e4b393f1aa97 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:02.134094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f489185-4fdf-4d60-83fb-f84f746a774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:02.134736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f489185-4fdf-4d60-83fb-f84f746a774a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.135013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f489185-4fdf-4d60-83fb-f84f746a774a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.135313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f489185-4fdf-4d60-83fb-f84f746a774a None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.136836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f489185-4fdf-4d60-83fb-f84f746a774a None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:02.138305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f489185-4fdf-4d60-83fb-f84f746a774a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:02.140027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 149/293] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:01 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:02.146104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:02.206862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc2e03-7664-4bd6-959b-d9856749081a None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.207214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc2e03-7664-4bd6-959b-d9856749081a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.207434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc2e03-7664-4bd6-959b-d9856749081a None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.208943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 150/294] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:02.214162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c8ac0-422d-4e77-a695-5108505b1800 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.215203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c8ac0-422d-4e77-a695-5108505b1800 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.215526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c8ac0-422d-4e77-a695-5108505b1800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.236596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e68c8ac0-422d-4e77-a695-5108505b1800 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:02.238825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e68c8ac0-422d-4e77-a695-5108505b1800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:02.279119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb599d-1b17-4e6e-97fc-97ccd186e437 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:02.340151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:02.351838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6186-9a4c-475c-9a2e-fc1b39c48af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:02.352470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6186-9a4c-475c-9a2e-fc1b39c48af3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.352752 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6186-9a4c-475c-9a2e-fc1b39c48af3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.353031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6186-9a4c-475c-9a2e-fc1b39c48af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.353768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6186-9a4c-475c-9a2e-fc1b39c48af3 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=3d5404b4f65a45f7b5c67c52ad140ef2) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:02.354077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6186-9a4c-475c-9a2e-fc1b39c48af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:02.362578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6186-9a4c-475c-9a2e-fc1b39c48af3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:02.366302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 151/295] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:02.372376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 145/296] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:02 2022] GET /identity/v3/role_assignments?user.id=3d5404b4f65a45f7b5c67c52ad140ef2&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:02.638976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447257a-5cab-44f3-b7b8-3d0bc6fa7c7f None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.639530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447257a-5cab-44f3-b7b8-3d0bc6fa7c7f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.640031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447257a-5cab-44f3-b7b8-3d0bc6fa7c7f None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.642881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 152/297] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:02.650912 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8072-b97c-49a1-8ce1-35cd7236912f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.651432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8072-b97c-49a1-8ce1-35cd7236912f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.652600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8072-b97c-49a1-8ce1-35cd7236912f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.679047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-5daf8072-b97c-49a1-8ce1-35cd7236912f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:02.680311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-5daf8072-b97c-49a1-8ce1-35cd7236912f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:02.712786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:02.719322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447257a-5cab-44f3-b7b8-3d0bc6fa7c7f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:02.766476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 146/298] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:02.784031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed471d3-49ec-4114-88db-e6fd365979f3 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.784387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed471d3-49ec-4114-88db-e6fd365979f3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.784767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed471d3-49ec-4114-88db-e6fd365979f3 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.786609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 147/299] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:54:02.794633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3745-c54b-425e-8885-efcd40d674f9 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.794919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3745-c54b-425e-8885-efcd40d674f9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.795333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3745-c54b-425e-8885-efcd40d674f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.826493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0d1c3745-c54b-425e-8885-efcd40d674f9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:02.827734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0d1c3745-c54b-425e-8885-efcd40d674f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:02.853231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447257a-5cab-44f3-b7b8-3d0bc6fa7c7f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:02.906007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:02.907053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:02.907445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:02.907721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:02.923099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:02.928708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:02.929626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:02.930416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:02.933908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 153/300] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:02.939049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 148/301] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:02.940423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:02.996621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:03.050269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318e806-2919-4278-9342-9596e8fea061 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.088221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.088843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.089189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.090246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.091634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:03.092169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:03.094735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:03.108578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 154/302] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:03.114191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c3745-c54b-425e-8885-efcd40d674f9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:03.126511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:03.250530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:03.256029 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4093c2-31c9-4a04-9e42-b3497708ce8f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.261879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c3745-c54b-425e-8885-efcd40d674f9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.270417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:03.312461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.313103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.313384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.313655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.314765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:03.316245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:03.318003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 155/303] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:03.323307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:03.329878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.330915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.331427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.331899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] PATH_INFO: `/v3/users/aab15d18dd384bd29d19ceb807293d1c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.360540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] RBAC: Authorizing `identity:get_user(user_id=aab15d18dd384bd29d19ceb807293d1c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:03.362042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:03.367427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 149/304] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:54:03 2022] GET /identity/v3/users/aab15d18dd384bd29d19ceb807293d1c => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:03.368895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:03.455022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93e101-4765-4e0b-95fa-53ff06c70d8a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.486124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107098e8-df69-48d7-85dd-7fad372357a6 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.493123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.493768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.494053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.494324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.506706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:03.513436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:03.515493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 156/305] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:03.517077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:03.534216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1c10d-0f86-4706-adf5-c9a899820fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.535217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1c10d-0f86-4706-adf5-c9a899820fc4 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.535892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1c10d-0f86-4706-adf5-c9a899820fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.536343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1c10d-0f86-4706-adf5-c9a899820fc4 None admin] PATH_INFO: `/v3/groups` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.537213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1c10d-0f86-4706-adf5-c9a899820fc4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:03.539766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1c10d-0f86-4706-adf5-c9a899820fc4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:03.561049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 150/306] 10.222.0.94 () {60 vars in 1134 bytes} [Mon Aug 29 18:54:03 2022] POST /identity/v3/groups => generated 229 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:03.562183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ed37-f4a8-4831-93f2-ff69ea5d8ccd None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.562475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ed37-f4a8-4831-93f2-ff69ea5d8ccd None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.562749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ed37-f4a8-4831-93f2-ff69ea5d8ccd None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.564043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 151/307] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:03.564814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944ed37-f4a8-4831-93f2-ff69ea5d8ccd None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:03.583728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:03.607035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:03.681899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab4ab5-1783-4baf-842b-b03d18648798 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.725201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b99dea-3744-41d9-bb5c-9a79b7de878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.726183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b99dea-3744-41d9-bb5c-9a79b7de878a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.726462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b99dea-3744-41d9-bb5c-9a79b7de878a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.726725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b99dea-3744-41d9-bb5c-9a79b7de878a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.727508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b99dea-3744-41d9-bb5c-9a79b7de878a None admin] RBAC: Adding query filter params (role.id=d5d02044c9984ae2ba7ebf348711f5ba, user.id=aab15d18dd384bd29d19ceb807293d1c, scope.system=all) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:03.727820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b99dea-3744-41d9-bb5c-9a79b7de878a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:03.729269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b99dea-3744-41d9-bb5c-9a79b7de878a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:03.736288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 157/308] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Aug 29 18:54:03 2022] GET /identity/v3/role_assignments?user.id=aab15d18dd384bd29d19ceb807293d1c&scope.system=all&role.id=d5d02044c9984ae2ba7ebf348711f5ba => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:03.737697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bd87-4788-4974-adab-3a7bd5572dce None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.737993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bd87-4788-4974-adab-3a7bd5572dce None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.738268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bd87-4788-4974-adab-3a7bd5572dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.752949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944ed37-f4a8-4831-93f2-ff69ea5d8ccd None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.760631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7371bd87-4788-4974-adab-3a7bd5572dce None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:03.762862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7371bd87-4788-4974-adab-3a7bd5572dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:03.809868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.810502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.810782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.811055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.812483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:03.812944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:03.814617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:03.825250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 152/309] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:03.831109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:03.873309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 158/310] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:03.931781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:03.957021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b9ed-618f-4b9a-bfe6-f08c1070f881 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:03.996929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:03.997583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:03.997913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:03.998192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] PATH_INFO: `/v3/users/e8982d50fbca413f8a275726631c1131` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:03.999514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] RBAC: Authorizing `identity:get_user(user_id=e8982d50fbca413f8a275726631c1131)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:04.000968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:04.002897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 153/311] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:54:03 2022] GET /identity/v3/users/e8982d50fbca413f8a275726631c1131 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:04.008559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bd87-4788-4974-adab-3a7bd5572dce None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:04.053973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:04.134423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bd87-4788-4974-adab-3a7bd5572dce None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:04.160615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:04.186076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:04.186986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:04.187407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:04.187784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] PATH_INFO: `/v3/system/users/e8982d50fbca413f8a275726631c1131/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:04.191537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e8982d50fbca413f8a275726631c1131, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:04.195437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:04.211198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 159/312] 10.222.0.94 () {58 vars in 1334 bytes} [Mon Aug 29 18:54:04 2022] PUT /identity/v3/system/users/e8982d50fbca413f8a275726631c1131/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:04.310691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a666f73-faa3-49d6-b8b3-0bbb1547cf95 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:04.352503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:04.353225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:04.353531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:04.353744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:04.367002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:04.374537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:04.375547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:04.376341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 154/313] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:04.383157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:04.512132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:04.522246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201f46-41a4-4df8-b2e8-02568a55a817 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:04.583501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:04.584432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:04.584914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:04.585387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:04.586639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:04.587196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:04.590409 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:04.606968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 160/314] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:04.612194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:04.618016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:04.745186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedbb1d-ffcb-496b-9dfe-8f45e811d9f5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:04.791550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:04.793000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:04.793483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:04.793904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] PATH_INFO: `/v3/users/acf2e53755af4e7ea93a6e1e88dd8303` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:04.795324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] RBAC: Authorizing `identity:get_user(user_id=acf2e53755af4e7ea93a6e1e88dd8303)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:04.797026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:04.798868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 155/315] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:54:04 2022] GET /identity/v3/users/acf2e53755af4e7ea93a6e1e88dd8303 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:04.805511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:04.929904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aaa03-4d85-40b2-a803-784fe6db4cdf None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:04.973657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:04.974407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:04.974758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:04.975117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:05.017656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:05.065252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:05.066822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:05.175626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 161/316] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:54:05.181039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:05.265165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:05.299730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f5c2-fe89-477e-b885-f9255094072e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:05.334592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e9f39-9d6c-4b31-b933-ed374bb2f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:05.335354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9f39-9d6c-4b31-b933-ed374bb2f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:05.335705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9f39-9d6c-4b31-b933-ed374bb2f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:05.336047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9f39-9d6c-4b31-b933-ed374bb2f9b1 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:05.337242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9f39-9d6c-4b31-b933-ed374bb2f9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:05.338743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9f39-9d6c-4b31-b933-ed374bb2f9b1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:05.341017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 156/317] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:05 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:05.348071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:05.414065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:05.463326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4382ecb-dbd9-4b34-953e-e5fbe05aeaf2 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:05.503446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d60c6-b134-449d-b226-5e4776382967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:05.504336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d60c6-b134-449d-b226-5e4776382967 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:05.504750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d60c6-b134-449d-b226-5e4776382967 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:05.505174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d60c6-b134-449d-b226-5e4776382967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:05.506106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d60c6-b134-449d-b226-5e4776382967 None admin] RBAC: Adding query filter params (role.id=d5d02044c9984ae2ba7ebf348711f5ba, scope.project.id=f73f010330ab41eead750a6cbd54038f, user.id=acf2e53755af4e7ea93a6e1e88dd8303) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:05.506584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d60c6-b134-449d-b226-5e4776382967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:05.508269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d60c6-b134-449d-b226-5e4776382967 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:05.516185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 162/318] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:05 2022] GET /identity/v3/role_assignments?user.id=acf2e53755af4e7ea93a6e1e88dd8303&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=d5d02044c9984ae2ba7ebf348711f5ba => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:05.828502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5fd08-89a9-47b1-8586-4a9b284ae9ab None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:05.829050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5fd08-89a9-47b1-8586-4a9b284ae9ab None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:05.829050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5fd08-89a9-47b1-8586-4a9b284ae9ab None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:05.830390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 157/319] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:05.838430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec018-257c-4a37-93ec-332ee65b51d5 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:05.838724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec018-257c-4a37-93ec-332ee65b51d5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:05.839010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec018-257c-4a37-93ec-332ee65b51d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:05.848534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184106-ad00-4340-a405-199d32b3445c None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:05.848824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184106-ad00-4340-a405-199d32b3445c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:05.849084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184106-ad00-4340-a405-199d32b3445c None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:05.850722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 158/320] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:54:05.856335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfc36-e166-4abe-9ea6-b4f0bdcfa4ef None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:05.856617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfc36-e166-4abe-9ea6-b4f0bdcfa4ef None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:05.856889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfc36-e166-4abe-9ea6-b4f0bdcfa4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:05.859161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c5bec018-257c-4a37-93ec-332ee65b51d5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:05.860392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c5bec018-257c-4a37-93ec-332ee65b51d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:05.875996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-530dfc36-e166-4abe-9ea6-b4f0bdcfa4ef None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:05.877184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-530dfc36-e166-4abe-9ea6-b4f0bdcfa4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:05.947192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 159/321] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:05.949418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 163/322] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:06.226781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfc36-e166-4abe-9ea6-b4f0bdcfa4ef None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:06.226974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec018-257c-4a37-93ec-332ee65b51d5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:06.289084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:06.312704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:06.351557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec018-257c-4a37-93ec-332ee65b51d5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:06.359131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfc36-e166-4abe-9ea6-b4f0bdcfa4ef None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:06.390813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336a9be-24f3-4591-958c-45e726115d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:06.391491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a9be-24f3-4591-958c-45e726115d37 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.391769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a9be-24f3-4591-958c-45e726115d37 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.392047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a9be-24f3-4591-958c-45e726115d37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.396957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b766d-e060-43ee-b73a-fbc8299a628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:06.397597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b766d-e060-43ee-b73a-fbc8299a628d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.397878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b766d-e060-43ee-b73a-fbc8299a628d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.398150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b766d-e060-43ee-b73a-fbc8299a628d None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.399298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b766d-e060-43ee-b73a-fbc8299a628d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:06.400745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b766d-e060-43ee-b73a-fbc8299a628d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:06.403180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 160/323] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:06.404493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88a5b-25e2-49b1-ab44-07041e1bbfa7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.404889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88a5b-25e2-49b1-ab44-07041e1bbfa7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.405484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88a5b-25e2-49b1-ab44-07041e1bbfa7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.406659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 161/324] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:06.411059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336a9be-24f3-4591-958c-45e726115d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:06.411494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d88a5b-25e2-49b1-ab44-07041e1bbfa7 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-c336a9be-24f3-4591-958c-45e726115d37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:06.417147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:06.419273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:06.420853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 164/325] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:06.422151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f863ca-c24f-42cb-a582-17f8c60d0ff4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.422151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f863ca-c24f-42cb-a582-17f8c60d0ff4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.422532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f863ca-c24f-42cb-a582-17f8c60d0ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.442818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e5f863ca-c24f-42cb-a582-17f8c60d0ff4 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:06.444071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e5f863ca-c24f-42cb-a582-17f8c60d0ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:06.529675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 165/326] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:06.530535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f863ca-c24f-42cb-a582-17f8c60d0ff4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:06.547535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d88a5b-25e2-49b1-ab44-07041e1bbfa7 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:06.585843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:06.587045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:06.587639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.587902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.588172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] PATH_INFO: `/v3/groups` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.589052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:06.591768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:06.597354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:06.615245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 162/327] 10.222.0.94 () {60 vars in 1134 bytes} [Mon Aug 29 18:54:06 2022] POST /identity/v3/groups => generated 226 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:06.666448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f863ca-c24f-42cb-a582-17f8c60d0ff4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:06.711951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:06.712782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.713260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.713410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.714173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:06.714487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:06.715935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:06.727320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 166/328] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:06.733140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:06.806299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:06.872626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc9f75-12b3-49bb-a2ce-00123b135634 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:06.916232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:06.918048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.918830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.919372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] PATH_INFO: `/v3/users/aab15d18dd384bd29d19ceb807293d1c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.920967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] RBAC: Authorizing `identity:get_user(user_id=aab15d18dd384bd29d19ceb807293d1c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:06.923449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:06.925941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 163/329] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:54:06 2022] GET /identity/v3/users/aab15d18dd384bd29d19ceb807293d1c => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:06.931575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:06.932471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20589e-d629-432b-bf64-1f826dd570bf None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:06.936854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:06.976558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:06.977300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:06.977576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:06.977845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:06.992178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:06.997927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:06.999291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:07.000084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:07.002397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 167/330] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:07.008050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:07.021454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:07.075552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd343cc-e8cf-49c9-b80f-ca9ec4d27605 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:07.122874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:07.124195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:07.124478 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:07.124756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] PATH_INFO: `/v3/system/users/aab15d18dd384bd29d19ceb807293d1c/roles/d5d02044c9984ae2ba7ebf348711f5ba` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:07.127303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aab15d18dd384bd29d19ceb807293d1c, role_id=d5d02044c9984ae2ba7ebf348711f5ba)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:07.129339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:07.141952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 164/331] 10.222.0.94 () {58 vars in 1334 bytes} [Mon Aug 29 18:54:06 2022] PUT /identity/v3/system/users/aab15d18dd384bd29d19ceb807293d1c/roles/d5d02044c9984ae2ba7ebf348711f5ba => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:54:07.161925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570dcda-07eb-4fa0-be19-af2a39a67576 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:07.204064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:07.204743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:07.205031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:07.205309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:07.206017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:07.206432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:07.207913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:07.218494 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 168/332] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:07.224111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:07.338651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5085d-38bd-41d4-b056-b8a41984e83b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:07.367651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:07.376618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5f755-dc37-42b5-9d5e-b07fb1dcfb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:07.377285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5f755-dc37-42b5-9d5e-b07fb1dcfb49 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:07.377591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5f755-dc37-42b5-9d5e-b07fb1dcfb49 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:07.377868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5f755-dc37-42b5-9d5e-b07fb1dcfb49 None admin] PATH_INFO: `/v3/users/e8982d50fbca413f8a275726631c1131` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:07.379227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5f755-dc37-42b5-9d5e-b07fb1dcfb49 None admin] RBAC: Authorizing `identity:get_user(user_id=e8982d50fbca413f8a275726631c1131)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:07.381250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5f755-dc37-42b5-9d5e-b07fb1dcfb49 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:07.386687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 165/333] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:54:07 2022] GET /identity/v3/users/e8982d50fbca413f8a275726631c1131 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:07.389807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:07.433901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:07.543791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147c3a3-fbd6-4c4b-bab7-cc4981bc5321 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:07.595630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f9e86-4ab5-4ff8-94de-be2493bf4f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:07.596177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f9e86-4ab5-4ff8-94de-be2493bf4f27 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:07.596464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f9e86-4ab5-4ff8-94de-be2493bf4f27 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:07.596749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f9e86-4ab5-4ff8-94de-be2493bf4f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:07.597485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f9e86-4ab5-4ff8-94de-be2493bf4f27 None admin] RBAC: Adding query filter params (role.id=b2e24e1a6db641d3a56fde8d3b1d405c, user.id=e8982d50fbca413f8a275726631c1131, scope.system=all) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:07.597796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f9e86-4ab5-4ff8-94de-be2493bf4f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:07.599264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f9e86-4ab5-4ff8-94de-be2493bf4f27 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:07.613102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 169/334] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Aug 29 18:54:07 2022] GET /identity/v3/role_assignments?user.id=e8982d50fbca413f8a275726631c1131&scope.system=all&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:09.175978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f9bc-5d03-48e1-ab31-492fd9aa5c38 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:09.176904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f9bc-5d03-48e1-ab31-492fd9aa5c38 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:09.177385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f9bc-5d03-48e1-ab31-492fd9aa5c38 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:09.178953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 166/335] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:09.184755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e0cbf-dcad-411b-a606-f517c48597ad None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:09.185226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e0cbf-dcad-411b-a606-f517c48597ad None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:09.185299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e0cbf-dcad-411b-a606-f517c48597ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:09.212226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-411e0cbf-dcad-411b-a606-f517c48597ad None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:09.213607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-411e0cbf-dcad-411b-a606-f517c48597ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:09.282426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a9a3-acab-45ca-8bb6-c953419e8fd7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:09.283048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a9a3-acab-45ca-8bb6-c953419e8fd7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:09.283540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a9a3-acab-45ca-8bb6-c953419e8fd7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:09.285335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 167/336] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:09.290984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab89cf5-e658-4a4a-8e74-1c9a4a182992 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:09.291573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab89cf5-e658-4a4a-8e74-1c9a4a182992 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:09.292040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab89cf5-e658-4a4a-8e74-1c9a4a182992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:09.325433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 170/337] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:09.332649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-6ab89cf5-e658-4a4a-8e74-1c9a4a182992 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:09.334089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6ab89cf5-e658-4a4a-8e74-1c9a4a182992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:09.440644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 168/338] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 29 18:54:09.713795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e0cbf-dcad-411b-a606-f517c48597ad None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:09.731191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:09.739685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a64ac-01a9-4c88-8c55-b373341f0a83 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:09.739992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a64ac-01a9-4c88-8c55-b373341f0a83 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:09.740237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a64ac-01a9-4c88-8c55-b373341f0a83 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:09.741903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 169/339] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:09.749202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e6d43-d314-4108-9838-181aae0045b6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:09.749491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e6d43-d314-4108-9838-181aae0045b6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:09.749763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e6d43-d314-4108-9838-181aae0045b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:09.780178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-373e6d43-d314-4108-9838-181aae0045b6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:09.781400 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-373e6d43-d314-4108-9838-181aae0045b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:09.909352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e0cbf-dcad-411b-a606-f517c48597ad None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:09.949831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 170/340] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:09.978794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:09.980402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:09.980951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:09.981418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.004128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:10.009468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:10.010344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:10.011472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:10.014321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 171/341] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:10.021383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e6d43-d314-4108-9838-181aae0045b6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:10.108253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:10.111171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:10.182557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e6d43-d314-4108-9838-181aae0045b6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:10.216153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:10.260082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf77a-dcfd-4cb9-acf2-736e49f8248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:10.261711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf77a-dcfd-4cb9-acf2-736e49f8248f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.262153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf77a-dcfd-4cb9-acf2-736e49f8248f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.262433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf77a-dcfd-4cb9-acf2-736e49f8248f None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.263163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf77a-dcfd-4cb9-acf2-736e49f8248f None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:10.263483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf77a-dcfd-4cb9-acf2-736e49f8248f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:10.264951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf77a-dcfd-4cb9-acf2-736e49f8248f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:10.275647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 171/342] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:10.277895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef832dae-dcb1-40d1-be07-04dcbd5b8317 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.278441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef832dae-dcb1-40d1-be07-04dcbd5b8317 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.278890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef832dae-dcb1-40d1-be07-04dcbd5b8317 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.280744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 172/343] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:10.286133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef832dae-dcb1-40d1-be07-04dcbd5b8317 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:10.291280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfeafdd-18c0-42a1-9284-526e331bda26 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:10.375801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728ec78-6f7f-40b0-a636-def85adc397b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:10.376445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728ec78-6f7f-40b0-a636-def85adc397b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.376724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728ec78-6f7f-40b0-a636-def85adc397b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.376999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728ec78-6f7f-40b0-a636-def85adc397b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.395163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728ec78-6f7f-40b0-a636-def85adc397b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-f728ec78-6f7f-40b0-a636-def85adc397b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:10.401460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:10.403247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:10.404144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 172/344] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:54:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:10.405455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370c8e-410b-45cc-8db5-54d6a376de0a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.408273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370c8e-410b-45cc-8db5-54d6a376de0a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.409596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc370c8e-410b-45cc-8db5-54d6a376de0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.437380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bc370c8e-410b-45cc-8db5-54d6a376de0a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:10.438600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bc370c8e-410b-45cc-8db5-54d6a376de0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:10.462444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef832dae-dcb1-40d1-be07-04dcbd5b8317 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:10.506120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:10.506750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.507032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.507623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.508342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:10.546968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 173/345] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:10.548344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370c8e-410b-45cc-8db5-54d6a376de0a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:10.603180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:10.605114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:10.623497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:10.670654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc370c8e-410b-45cc-8db5-54d6a376de0a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:10.724536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:10.725451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.725987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.726403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.727400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:10.727860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:10.729547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:10.739493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 173/346] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:54:10.744009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:10.744268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 174/347] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:10.748403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:10.768249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:10.866664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0c99b-543c-4f18-9080-5204a339d858 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:10.867288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44ff14-6509-4da8-8b4c-112aec669646 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:10.894368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:10.904823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:10.905179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.905466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.905743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.910419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:10.911257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:10.911728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:10.912138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:10.924731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:10.930335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:10.931419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:10.932217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:10.933937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 174/348] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:10.936157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:10.985832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:10.987339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.085022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:11.098436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 175/349] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:11.099261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:11.121851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee44065-a12e-40c4-bac5-413bc4ceb89b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.131452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:11.176209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.176854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.177126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.177391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.178524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:11.180009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.181918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 175/350] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:10 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:11.182690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:11.218337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e7d4-9ae5-4528-bbcc-8de1e0500901 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.260782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.262258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.262258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.262258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.262907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:11.263247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:11.264635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.277397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 176/351] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:11.278498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:11.310319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2379744-a82f-459c-80f6-821d30c553ee None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.356371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.356807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.357145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.357544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.359616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:11.361077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.362798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 176/352] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:11.363575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:11.384259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:11.390831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:11.409465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750cf1a-fa77-410a-ae15-35091667718e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.456537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2514f9f-936d-497a-8ab8-f91fd759e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.456775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2514f9f-936d-497a-8ab8-f91fd759e75b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.457060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2514f9f-936d-497a-8ab8-f91fd759e75b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.457369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2514f9f-936d-497a-8ab8-f91fd759e75b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.472070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2514f9f-936d-497a-8ab8-f91fd759e75b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d2514f9f-936d-497a-8ab8-f91fd759e75b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:11.478106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:11.479011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:11.480588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 177/353] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:11.481381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adb4cd-a05e-43be-81ee-fcacace61b8b None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.481869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adb4cd-a05e-43be-81ee-fcacace61b8b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.482012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adb4cd-a05e-43be-81ee-fcacace61b8b None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.484052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 178/354] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:11.484052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adb4cd-a05e-43be-81ee-fcacace61b8b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:11.508550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bdb7a-60a2-4913-bdbe-7a7999a2a2e2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.552868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.553609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.553888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.554158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.561737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:11.563320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.564977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 177/355] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:11.565736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:11.618224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adb4cd-a05e-43be-81ee-fcacace61b8b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.668568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f26b38-d7be-4d36-9379-431bc43449b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.669312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26b38-d7be-4d36-9379-431bc43449b3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.669590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26b38-d7be-4d36-9379-431bc43449b3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.669858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26b38-d7be-4d36-9379-431bc43449b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.676239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:11.699410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:11.727760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f36ff1-099c-4d34-ac12-d35d0a847173 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.769044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f26b38-d7be-4d36-9379-431bc43449b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:11.770782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f26b38-d7be-4d36-9379-431bc43449b3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.776546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.777623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.778038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.778438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.780047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:11.782134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.784017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 178/356] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:11.784971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:11.938076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 179/357] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:11.938335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739543e0-58b8-4e2d-bb73-aba8cd975b4a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:11.939959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9e37a-535a-4985-8f5e-dfb8e21ad23e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.940433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9e37a-535a-4985-8f5e-dfb8e21ad23e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.940868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9e37a-535a-4985-8f5e-dfb8e21ad23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.966809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f5e9e37a-535a-4985-8f5e-dfb8e21ad23e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:11.968387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f5e9e37a-535a-4985-8f5e-dfb8e21ad23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:11.984406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:11.985032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:11.985357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:11.985634 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:11.986623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:11.987062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:11.989182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:11.998621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 179/358] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:12.000085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.015640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:12.042188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:12.074761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 180/359] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:12.075668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9e37a-535a-4985-8f5e-dfb8e21ad23e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.149186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced817f7-4403-4a1f-8279-e8f0bd23bc3a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.202562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f2445-3fd7-4f24-b114-e19a80346195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:12.203201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f2445-3fd7-4f24-b114-e19a80346195 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.203201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f2445-3fd7-4f24-b114-e19a80346195 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.206585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f2445-3fd7-4f24-b114-e19a80346195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.206585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f2445-3fd7-4f24-b114-e19a80346195 None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=581b1f65f92b42d9855b49a2423ba339) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:12.206585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f2445-3fd7-4f24-b114-e19a80346195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:12.206878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f2445-3fd7-4f24-b114-e19a80346195 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:12.214481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 180/360] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:11 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:12.214481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed043594-3c4a-4c0a-9793-cb74bee3dbc3 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.214481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed043594-3c4a-4c0a-9793-cb74bee3dbc3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.214859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed043594-3c4a-4c0a-9793-cb74bee3dbc3 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.216431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 181/361] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:12.218006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed043594-3c4a-4c0a-9793-cb74bee3dbc3 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.240301 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9e37a-535a-4985-8f5e-dfb8e21ad23e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.306821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:12.314461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:12.315597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.316160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.316640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.317628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=581b1f65f92b42d9855b49a2423ba339) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:12.318145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:12.319891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:12.326673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 181/362] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=377956ded99f409684f20afaaf7052c8 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:12.328102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.347039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed043594-3c4a-4c0a-9793-cb74bee3dbc3 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.391778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5a338-e6ec-4e80-bd99-47860a559ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:12.392435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5a338-e6ec-4e80-bd99-47860a559ce4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.392722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5a338-e6ec-4e80-bd99-47860a559ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.392999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5a338-e6ec-4e80-bd99-47860a559ce4 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.394199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5a338-e6ec-4e80-bd99-47860a559ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:12.396262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5a338-e6ec-4e80-bd99-47860a559ce4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:12.399095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 182/363] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:12.399095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43652438-fbbf-4308-a1d1-d891dd1c69af None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.399314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43652438-fbbf-4308-a1d1-d891dd1c69af None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.399570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43652438-fbbf-4308-a1d1-d891dd1c69af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.412347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:12.422445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-43652438-fbbf-4308-a1d1-d891dd1c69af None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:12.423737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-43652438-fbbf-4308-a1d1-d891dd1c69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:12.523285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d514c17-bb68-4fd4-9a55-c986be5c8a01 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.565125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 183/364] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:12.565320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43652438-fbbf-4308-a1d1-d891dd1c69af None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.575541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:12.576236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.576519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.576797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] PATH_INFO: `/v3/users/aab15d18dd384bd29d19ceb807293d1c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.578863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] RBAC: Authorizing `identity:get_user(user_id=aab15d18dd384bd29d19ceb807293d1c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:12.579588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:12.581627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 182/365] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/users/aab15d18dd384bd29d19ceb807293d1c => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:12.582419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.658323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:12.703020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43652438-fbbf-4308-a1d1-d891dd1c69af None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.722785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c196ec-f1b2-400b-9a96-8bc5e300dc80 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.746835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:12.767918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:12.768575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.768919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.769198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.771681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:12.774086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:12.779199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 184/366] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:12.779199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.805782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:12.806445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.806728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.807005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.835770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:12.842271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:12.843376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:12.844166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:12.851199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 183/367] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:12.852194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:12.944548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ab1f-6016-42f5-b990-30b3695d006a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.984794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8939aa3-9da1-48b9-ae32-d78bdf0378e0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:12.987882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:12.988512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:12.988791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:12.989064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:12.989803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] RBAC: Adding query filter params (role.id=d5d02044c9984ae2ba7ebf348711f5ba, user.id=aab15d18dd384bd29d19ceb807293d1c, scope.system=all) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:12.990108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:12.991634 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:13.003559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:13.006834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 185/368] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/role_assignments?user.id=aab15d18dd384bd29d19ceb807293d1c&scope.system=all&role.id=d5d02044c9984ae2ba7ebf348711f5ba => generated 510 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:13.012909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:13.028055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:13.028648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:13.028932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:13.029206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:13.029997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=f73f010330ab41eead750a6cbd54038f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:13.030308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:13.032949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:13.040081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 184/369] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:12 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:13.065800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:13.129224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:13.222950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd5364-3464-43e4-891b-4d913c7e70ee None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:13.227820 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ca1f-a4d9-4dcd-b2ce-e63d5675cbcc None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:13.265505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:13.267013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:13.267558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:13.268041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:13.284302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:13.290233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:13.291661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:13.292636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:13.295514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 185/370] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:13.296385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:13.297024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:13.297304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:13.297578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:13.298254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:13.298561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:13.303110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:13.304229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:13.315552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 186/371] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:13.332393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:13.421150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:13.472242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:13.488225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2780772-d527-41cf-9799-acf3ac2f1037 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:13.525894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b7ab0-6eff-4589-b35b-8ded5f606cd4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:13.544069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:13.544614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:13.544894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:13.545165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:13.545837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:13.546734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:13.548157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:13.567009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 186/372] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:13.572335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:13.595589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:13.596566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:13.597052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:13.597561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:13.733019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:13.734523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:13.748628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95b67f-245d-4e5b-9595-7fb27f29677e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:13.788829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:13.789864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:13.790385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:13.790855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:13.825542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:13.884739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:13.887625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:13.913785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:14.006231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 187/373] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:14.016915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:14.027117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 187/374] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:14.027802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:14.094963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:14.147767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31d56c-3af6-49bc-8d0e-62eb6a00086e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:14.187868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:14.188871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.189363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.189834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] PATH_INFO: `/v3/projects/11b2e035c0894ef7a5efbac9016e862c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:14.191208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] RBAC: Authorizing `identity:get_project(project_id=11b2e035c0894ef7a5efbac9016e862c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:14.192878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:14.194735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 188/375] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:14 2022] GET /identity/v3/projects/11b2e035c0894ef7a5efbac9016e862c => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:14.201570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:14.281690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2073e-ce65-45a0-970b-42002133ea86 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:14.361742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:14.362776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.363296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.363775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:14.366771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:14.369730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:14.374427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 188/376] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:14 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:14.380161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:14.439520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:14.446440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa5b01-b2d3-440e-8e1c-668b70a6b34b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:14.518920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e68f9-3276-494f-9326-a68f5e5e2451 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:14.536574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:14.568850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:14.569681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.570057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.570427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:14.571724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:14.573928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:14.576272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 189/377] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:14 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:14.582516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:14.584425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:14.585088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.585369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.585642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] PATH_INFO: `/v3/groups/36ba0cf741f84a1485acbf9ad24d985a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:14.596345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] RBAC: Authorizing `identity:get_group(group_id=36ba0cf741f84a1485acbf9ad24d985a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:14.597817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:14.600134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 189/378] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:14 2022] GET /identity/v3/groups/36ba0cf741f84a1485acbf9ad24d985a => generated 229 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:14.607753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:14.725181 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:14.736318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b34a8-2410-457d-9e1c-8e54da68cbc8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:14.790770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fa6ac-341c-402e-aadf-497c2f18921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:14.791601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa6ac-341c-402e-aadf-497c2f18921e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.791936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa6ac-341c-402e-aadf-497c2f18921e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.792338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa6ac-341c-402e-aadf-497c2f18921e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:14.793749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fa6ac-341c-402e-aadf-497c2f18921e None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=f73f010330ab41eead750a6cbd54038f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:14.794081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fa6ac-341c-402e-aadf-497c2f18921e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:14.796315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fa6ac-341c-402e-aadf-497c2f18921e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:14.803576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 190/379] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:14 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=377956ded99f409684f20afaaf7052c8 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:14.830139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87140c1-a5b6-49e1-8c95-42e4157b8711 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:14.889259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e443f95-6504-4110-9c50-e7bdb5700c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:14.889903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e443f95-6504-4110-9c50-e7bdb5700c37 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.890185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e443f95-6504-4110-9c50-e7bdb5700c37 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.890461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e443f95-6504-4110-9c50-e7bdb5700c37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:14.891243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e443f95-6504-4110-9c50-e7bdb5700c37 None admin] RBAC: Adding query filter params (group.id=36ba0cf741f84a1485acbf9ad24d985a, role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=11b2e035c0894ef7a5efbac9016e862c) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:14.891554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e443f95-6504-4110-9c50-e7bdb5700c37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:14.893037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e443f95-6504-4110-9c50-e7bdb5700c37 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:14.905351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 190/380] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:14 2022] GET /identity/v3/role_assignments?group.id=36ba0cf741f84a1485acbf9ad24d985a&scope.project.id=11b2e035c0894ef7a5efbac9016e862c&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 260 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:14.985481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9255b2e-d857-4bb1-a665-196bf8c44626 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.985780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9255b2e-d857-4bb1-a665-196bf8c44626 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.985958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9255b2e-d857-4bb1-a665-196bf8c44626 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:14.987563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 191/381] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:14.997877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da292e-a5ef-44df-a13d-2dc8a75fd6b7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:14.998208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da292e-a5ef-44df-a13d-2dc8a75fd6b7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:14.998447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da292e-a5ef-44df-a13d-2dc8a75fd6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.026340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c0da292e-a5ef-44df-a13d-2dc8a75fd6b7 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:15.027693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c0da292e-a5ef-44df-a13d-2dc8a75fd6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:15.115271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:15.141716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 191/382] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:15.420106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61cadb-3714-4108-9132-545dbbfdb29e None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:15.420620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61cadb-3714-4108-9132-545dbbfdb29e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:15.420620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61cadb-3714-4108-9132-545dbbfdb29e None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.422320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 192/383] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:15.428142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9399e3-ebad-4b21-affa-29bdafb0d379 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:15.428471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9399e3-ebad-4b21-affa-29bdafb0d379 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:15.428782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9399e3-ebad-4b21-affa-29bdafb0d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.455593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-af9399e3-ebad-4b21-affa-29bdafb0d379 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:15.456806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-af9399e3-ebad-4b21-affa-29bdafb0d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:15.548447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 192/384] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:15.577723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61cadb-3714-4108-9132-545dbbfdb29e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:15.712927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61cadb-3714-4108-9132-545dbbfdb29e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:15.762252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca76d26-db64-4ad3-91e7-062b8ffcf23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:15.762910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca76d26-db64-4ad3-91e7-062b8ffcf23d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:15.763221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca76d26-db64-4ad3-91e7-062b8ffcf23d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:15.763499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca76d26-db64-4ad3-91e7-062b8ffcf23d None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.777281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca76d26-db64-4ad3-91e7-062b8ffcf23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-aca76d26-db64-4ad3-91e7-062b8ffcf23d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:15.782663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:15.784808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:15.786715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:15.791097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 193/385] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:15.791345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9399e3-ebad-4b21-affa-29bdafb0d379 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:15.879572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140e064-883a-4cd2-b49c-f2e63f37e269 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:15.880189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140e064-883a-4cd2-b49c-f2e63f37e269 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:15.880662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140e064-883a-4cd2-b49c-f2e63f37e269 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.882206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 194/386] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:54:15.887950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e124da3-09cc-4781-870b-7cc6bdfe5612 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:15.888507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e124da3-09cc-4781-870b-7cc6bdfe5612 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:15.888992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e124da3-09cc-4781-870b-7cc6bdfe5612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:15.919470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-6e124da3-09cc-4781-870b-7cc6bdfe5612 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:15.926103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6e124da3-09cc-4781-870b-7cc6bdfe5612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:15.942986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:15.967672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9399e3-ebad-4b21-affa-29bdafb0d379 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:15.983447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.010822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 195/387] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:16.012521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e124da3-09cc-4781-870b-7cc6bdfe5612 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.019094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.019094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.019094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.019094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.019478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:16.019478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.019478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.030611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 193/388] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:16.040247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.166619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e73397-e78b-4fbf-99cd-6cac46e0cd00 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.209138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e124da3-09cc-4781-870b-7cc6bdfe5612 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.240713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.242187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.242638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.243390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.245025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.247331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.249541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 194/389] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:16.255200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.288631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.289291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.289623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.289917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.319281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:16.328692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:16.329650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:16.330431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:16.335458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 196/390] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:16.336721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.355606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.367846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.458550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d1732-93a4-47ed-acff-0db2c00d4150 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.459399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdccf2-7703-4872-b55c-2c957a5301aa None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.500512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.500856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.501549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.501669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.501831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.502108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.502373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.502779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.504081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.505987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.507975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 195/391] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:16.508953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.515365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:16.520380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:16.521268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:16.522058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 197/392] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:16.522695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.608020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.630380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d632a8e-ef74-458a-a933-e547c8faac9f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.644299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.651112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f1f9f-4cca-4f6f-8ff2-41f27ae192bf None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.672660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.673629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.674124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.674587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.675552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:16.676064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.677689 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.687654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 196/393] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:16.688815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.691808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.692545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.693977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.693977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.695600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=581b1f65f92b42d9855b49a2423ba339, group_id=ae519b35941040d3aee900a4f5ca4ad9, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.697074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.712500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 198/394] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Aug 29 18:54:16 2022] PUT /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:16.713520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.809153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684e8f2-f862-4e72-86b9-6fa30dea394a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.853090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb913-3456-4323-9f5a-584d9b2d36a1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:16.868284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.869696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.870997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.871709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.873157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:16.874063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.876706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.884728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:16.897522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:16.898449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:16.898860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:16.899298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:16.900676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:16.902878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:16.905126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 199/395] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:16.907093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 197/396] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:16.912443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:16.915237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:17.004582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:17.040855 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb42e-1c2b-4a11-8725-744b01533c55 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:17.081148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:17.082006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.082430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.082842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.085018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:17.086827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:17.089549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 200/397] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:17.094951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:17.134102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3feff4-25e0-4b2b-bf28-4a63e7a06032 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:17.166823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:17.194670 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:17.195679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.196235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.196691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.199408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:17.201526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:17.204034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 198/398] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:16 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:17.212135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:17.228327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b1ab2-f86a-42d4-a1bf-94bca2f6793b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:17.288576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:17.289540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.291037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.291486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.295758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=581b1f65f92b42d9855b49a2423ba339, group_id=ae519b35941040d3aee900a4f5ca4ad9, role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:17.300092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:17.325706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 201/399] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Aug 29 18:54:17 2022] PUT /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/377956ded99f409684f20afaaf7052c8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:17.365151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab50416-cdc1-4417-8f80-a5d706c749c9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:17.367030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:17.430898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b4bfd-7632-4981-85b1-646469891126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:17.431854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b4bfd-7632-4981-85b1-646469891126 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.432309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b4bfd-7632-4981-85b1-646469891126 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.433386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b4bfd-7632-4981-85b1-646469891126 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.438109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b4bfd-7632-4981-85b1-646469891126 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:17.439853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b4bfd-7632-4981-85b1-646469891126 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:17.441794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 199/400] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:17 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:17.447909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:17.462488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9248972-5193-4340-8bc9-44d644002ad9 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.463395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9248972-5193-4340-8bc9-44d644002ad9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.463804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9248972-5193-4340-8bc9-44d644002ad9 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.465900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 200/401] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:17.482042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422220f-6078-422d-b409-57282d67e22d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.482677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422220f-6078-422d-b409-57282d67e22d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.483183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422220f-6078-422d-b409-57282d67e22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.516969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c422220f-6078-422d-b409-57282d67e22d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:17.518645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c422220f-6078-422d-b409-57282d67e22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:17.570398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69c2b7-562d-4809-900b-5f01f6feadb4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:17.606338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 201/402] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:17.608148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6070e16-3707-4d34-b2d3-8f39f5d41d7a None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.608559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6070e16-3707-4d34-b2d3-8f39f5d41d7a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.609020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6070e16-3707-4d34-b2d3-8f39f5d41d7a None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.614636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 202/403] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:17.616398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c695cab-65ea-4b95-8f24-2ae8ea5cec27 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.616731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c695cab-65ea-4b95-8f24-2ae8ea5cec27 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.617109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c695cab-65ea-4b95-8f24-2ae8ea5cec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.621928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:17.643667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-4c695cab-65ea-4b95-8f24-2ae8ea5cec27 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:17.645041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4c695cab-65ea-4b95-8f24-2ae8ea5cec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:17.662328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:17.663085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:17.663689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:17.663976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:17.666689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f73f010330ab41eead750a6cbd54038f, group_id=ae519b35941040d3aee900a4f5ca4ad9, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:17.668254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:17.685176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 202/404] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Aug 29 18:54:17 2022] PUT /identity/v3/projects/f73f010330ab41eead750a6cbd54038f/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:17.747268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 203/405] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:18.049922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:18.055895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c695cab-65ea-4b95-8f24-2ae8ea5cec27 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:18.110685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:18.191045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c695cab-65ea-4b95-8f24-2ae8ea5cec27 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:18.229047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:18.230164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.230814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.231263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.242649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:18.248554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:18.249549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:18.250345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:18.252651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 204/406] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:18.257883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:18.338046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b02c0-e4fa-4a6c-b30b-230e73244eb0 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:18.394834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:18.404071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4d977-f454-4d28-bc3e-a341cc626c96 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:18.443149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:18.444176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.444615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.445100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.445940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:18.446388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:18.448062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:18.459489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 205/407] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:18.466009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:18.467103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.467595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.468069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.472166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:18.492401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:18.499697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:18.500605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:18.501382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:18.504328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 203/408] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:18.512645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:18.595615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:18.643557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8426a-c9f4-48cc-aec6-c9c84726dd39 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:18.682317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:18.683730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.684216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.684684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.685730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:18.686258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:18.687874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:18.698879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 204/409] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:18.705275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:18.829170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220aa49e-033f-4c7f-b9a6-016cdf0af664 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:18.844401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f1363-977d-46f7-9e45-32eecc41b02f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:18.858165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:18.905031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:18.906051 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.906550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.907042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.910560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:18.913184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:18.916167 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 206/410] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:18.922228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:18.949932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:18.951195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:18.951481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:18.951775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] PATH_INFO: `/v3/groups/36ba0cf741f84a1485acbf9ad24d985a` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:18.958824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] RBAC: Authorizing `identity:get_group(group_id=36ba0cf741f84a1485acbf9ad24d985a)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:18.960330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:18.961979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 205/411] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/groups/36ba0cf741f84a1485acbf9ad24d985a => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:18.962419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:18.968911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:19.060507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aff7ee-73a6-47e8-a35c-8554b00521f6 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:19.109376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:19.109945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.110227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.110505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.111446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1269-c07a-4298-9e5f-48f2a5d635ff None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:19.111737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:19.113260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:19.119499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 207/412] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:19.122158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:19.186184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:19.187330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.187842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.188313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] PATH_INFO: `/v3/projects/11b2e035c0894ef7a5efbac9016e862c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.195817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] RBAC: Authorizing `identity:get_project(project_id=11b2e035c0894ef7a5efbac9016e862c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:19.197588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:19.200160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 206/413] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:18 2022] GET /identity/v3/projects/11b2e035c0894ef7a5efbac9016e862c => generated 344 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:19.206152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:19.229199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:19.243285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a54900-7862-4c86-ab0d-a1c5e6c7bd05 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:19.286310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a5969-846a-49ed-b74b-f973918aa45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:19.286988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a5969-846a-49ed-b74b-f973918aa45c None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.287306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a5969-846a-49ed-b74b-f973918aa45c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.287587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a5969-846a-49ed-b74b-f973918aa45c None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.292470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a5969-846a-49ed-b74b-f973918aa45c None admin] RBAC: Authorizing `identity:create_grant(project_id=f73f010330ab41eead750a6cbd54038f, group_id=ae519b35941040d3aee900a4f5ca4ad9, role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:19.294423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a5969-846a-49ed-b74b-f973918aa45c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:19.318420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:19.318607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 208/414] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Aug 29 18:54:19 2022] PUT /identity/v3/projects/f73f010330ab41eead750a6cbd54038f/groups/ae519b35941040d3aee900a4f5ca4ad9/roles/377956ded99f409684f20afaaf7052c8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:19.402794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb616-712b-43df-92a0-a1d5a105c72a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:19.449985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49576be5-14bc-45e2-8481-c8bcf999de62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:19.450665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49576be5-14bc-45e2-8481-c8bcf999de62 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.450955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49576be5-14bc-45e2-8481-c8bcf999de62 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.451263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49576be5-14bc-45e2-8481-c8bcf999de62 None admin] PATH_INFO: `/v3/projects/11b2e035c0894ef7a5efbac9016e862c/groups/36ba0cf741f84a1485acbf9ad24d985a/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.453245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49576be5-14bc-45e2-8481-c8bcf999de62 None admin] RBAC: Authorizing `identity:create_grant(project_id=11b2e035c0894ef7a5efbac9016e862c, group_id=36ba0cf741f84a1485acbf9ad24d985a, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:19.454767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49576be5-14bc-45e2-8481-c8bcf999de62 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:19.472732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 207/415] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Aug 29 18:54:19 2022] PUT /identity/v3/projects/11b2e035c0894ef7a5efbac9016e862c/groups/36ba0cf741f84a1485acbf9ad24d985a/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:54:19.503503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054985-ead8-4996-91a6-18c5cf4be07d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.504906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054985-ead8-4996-91a6-18c5cf4be07d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.505420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c054985-ead8-4996-91a6-18c5cf4be07d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.507518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 209/416] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:19.514733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cad27-8546-4897-9c47-d869b446c0d3 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:19.515149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cad27-8546-4897-9c47-d869b446c0d3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:19.515314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cad27-8546-4897-9c47-d869b446c0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:19.538108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1f6cad27-8546-4897-9c47-d869b446c0d3 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:19.539545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1f6cad27-8546-4897-9c47-d869b446c0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:19.633121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 208/417] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:19.937003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c054985-ead8-4996-91a6-18c5cf4be07d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:20.090357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c054985-ead8-4996-91a6-18c5cf4be07d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:20.097847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648faa-6787-4609-bff1-ecdcb8c2d315 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.098629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648faa-6787-4609-bff1-ecdcb8c2d315 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.099415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648faa-6787-4609-bff1-ecdcb8c2d315 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.102438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 209/418] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:20.112258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b505d8-cfcb-4857-9ab7-475222d6ae6d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.113031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b505d8-cfcb-4857-9ab7-475222d6ae6d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.113722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b505d8-cfcb-4857-9ab7-475222d6ae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.157314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-13b505d8-cfcb-4857-9ab7-475222d6ae6d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:20.158782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-13b505d8-cfcb-4857-9ab7-475222d6ae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:20.168617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:20.168617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.168617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.168617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.189784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:20.195474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:20.196585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:20.197551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:20.204310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 210/419] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:20.210772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:20.218511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:20.230198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:20.285302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 210/420] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:20.287114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa57f3-d835-4d8b-93d5-9dd5530c7b70 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.287586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa57f3-d835-4d8b-93d5-9dd5530c7b70 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.288040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa57f3-d835-4d8b-93d5-9dd5530c7b70 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.289547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 211/421] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:20.296762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576721eb-ef64-4850-a76d-b717dc8c2b49 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.297379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576721eb-ef64-4850-a76d-b717dc8c2b49 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.297839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576721eb-ef64-4850-a76d-b717dc8c2b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.322278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-576721eb-ef64-4850-a76d-b717dc8c2b49 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:20.324921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-576721eb-ef64-4850-a76d-b717dc8c2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:20.382673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e988aa-8bae-419e-9580-f4e63c7ec7d2 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:20.408922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 212/422] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:20.428079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:20.428768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.429046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.429312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.429984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] RBAC: Adding query filter params (name=member) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:20.430285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:20.431731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:20.444090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 211/423] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:20.450242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576721eb-ef64-4850-a76d-b717dc8c2b49 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:20.562184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:20.572748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576721eb-ef64-4850-a76d-b717dc8c2b49 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:20.611795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:20.612734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:20.613257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:20.613774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:20.687860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:20.690049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:20.787974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:20.790508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:20.810314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 213/424] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:20.820445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:20.930719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:20.977954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e929d0-c8b9-487b-ab3b-a74e1a1bd76e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:20.983993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0884a4-e32e-4f78-af4b-9df6466aa55e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.031635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.032637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.033069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.033486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.033808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.034546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.034947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.035321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:21.035431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.037286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:21.044689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 214/425] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:20 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:21.046624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.051337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:21.056491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:21.057410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 18:54:21.058930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:21.063708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 212/426] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:21.065384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.115484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.166338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac4e7-1dc2-4346-8c4a-ef1d8b30af59 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.204500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.205509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.205966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.206497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.217685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:21.223614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:21.224519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:21.225326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:21.227690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 215/427] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:21.228747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.267796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c3c1c-37bb-4700-a9a5-134764ce5c45 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.323212 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.333255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.334597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.335389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.336046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.338198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:21.341909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:21.342989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f77719-cc88-4405-bfe9-a64aab8a40e1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.346092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 213/428] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:21.347495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.366614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.382294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.383237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.383676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.384105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.384944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] RBAC: Adding query filter params (name=member) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:21.385419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:21.387580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:21.397256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 216/429] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:21.398496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.508458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051383-719c-4944-b73c-7cb0f3f1dd32 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.555578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.556565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.556889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b9b83-be6d-4f01-9474-c8a3760b2b62 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.557255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.557846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.559532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=581b1f65f92b42d9855b49a2423ba339) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:21.560143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:21.561975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:21.568815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 217/430] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:21.569932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.625724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.628102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.629895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.630576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.631877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.633880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:21.634640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:21.637674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:21.655749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 214/431] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:21.668041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.700166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ed7fe-1d3e-4d83-b581-b2ca1edbb3ab None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.738976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.739755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.740232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.740380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:21.827119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:21.828732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:21.863154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.884067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:21.925412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ac6f2-0da9-4f5d-8a54-7b76effcd472 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:21.945227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 218/432] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:21.952475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:21.974866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:21.975850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:21.976350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:21.976819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.069333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da2deb-f89e-4e4e-a135-756cd601902a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.096180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.098764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:22.109807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37985116-4f2d-4e63-8d66-1f5ff1a1b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.110456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985116-4f2d-4e63-8d66-1f5ff1a1b87a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.110735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985116-4f2d-4e63-8d66-1f5ff1a1b87a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.111010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985116-4f2d-4e63-8d66-1f5ff1a1b87a None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.112190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37985116-4f2d-4e63-8d66-1f5ff1a1b87a None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.113658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37985116-4f2d-4e63-8d66-1f5ff1a1b87a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:22.115588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 219/433] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:22.116590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be9db-63c9-43bf-b4d6-cf930ce59d5b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.116880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be9db-63c9-43bf-b4d6-cf930ce59d5b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.117156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be9db-63c9-43bf-b4d6-cf930ce59d5b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.118598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 220/434] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:22.119551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15b284-fa14-4339-9709-afcf931571c7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.119828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15b284-fa14-4339-9709-afcf931571c7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.120252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15b284-fa14-4339-9709-afcf931571c7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.121556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 221/435] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:22.126757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c956b54-ba01-40df-b97d-712ad5650e94 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.127267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c956b54-ba01-40df-b97d-712ad5650e94 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.127458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c956b54-ba01-40df-b97d-712ad5650e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.148117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8c956b54-ba01-40df-b97d-712ad5650e94 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:22.149313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8c956b54-ba01-40df-b97d-712ad5650e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:22.160893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.194798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.243383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 222/436] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:22.249846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 215/437] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:22.249994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358ad78-cd21-40c6-8fae-e22a51606841 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.250084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.250166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358ad78-cd21-40c6-8fae-e22a51606841 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.250166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358ad78-cd21-40c6-8fae-e22a51606841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.269744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4358ad78-cd21-40c6-8fae-e22a51606841 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:22.271196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4358ad78-cd21-40c6-8fae-e22a51606841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:22.345929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 223/438] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:22.346636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358ad78-cd21-40c6-8fae-e22a51606841 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.376769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6f99-43e0-439b-8c2e-e6e4dcdb16ca None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.419557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.420352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.420733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.421110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.422356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.424106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:22.432471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 216/439] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:22.433485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.433815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.464089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358ad78-cd21-40c6-8fae-e22a51606841 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.465638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.503297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.504183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.504610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.505028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] PATH_INFO: `/v3/projects/581b1f65f92b42d9855b49a2423ba339` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.506378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] RBAC: Authorizing `identity:get_project(project_id=581b1f65f92b42d9855b49a2423ba339)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.508113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:22.515276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 224/440] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/projects/581b1f65f92b42d9855b49a2423ba339 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:22.517367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.560194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d02887-5114-4421-a7e4-82a4a767e6b1 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.598342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.598916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.599279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.599630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.600218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=b2e24e1a6db641d3a56fde8d3b1d405c, scope.project.id=f73f010330ab41eead750a6cbd54038f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:22.600524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.601954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:22.608273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 217/441] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=b2e24e1a6db641d3a56fde8d3b1d405c => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:22.620633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.633982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cd7a8-ec29-4048-b1e9-dfc549b299ba None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.682894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.683934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.684362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.684774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.697023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:22.704189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:22.705757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:22.706730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:22.713759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 225/442] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:22.715760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.718901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.747290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.761906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48080b7b-d133-4414-9cf0-e5ca3b3f011f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.812471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.813109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.813390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.813661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.814896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.817127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:22.824226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 218/443] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:22.824226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.848631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9350f54-0836-4830-a1c0-ca8424890334 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.883749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.884381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.884663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.884932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.898714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:22.907968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:22.909029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:22.909924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 226/444] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:22.911043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:22.950558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9d8c6-dc1f-4437-a6a5-b4d3509290b0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:22.992841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:22.993450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:22.993728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:22.993997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:22.994681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:22.994965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:22.996178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:22.996909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.005679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 219/445] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:23.006509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.031810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:23.054158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1502e-0adb-4f39-bf49-0a0954805220 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.090126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.090743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.091016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.091316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.092036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=581b1f65f92b42d9855b49a2423ba339) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:23.092340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.093784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.100194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 227/446] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:22 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=581b1f65f92b42d9855b49a2423ba339&role.id=377956ded99f409684f20afaaf7052c8 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:23.100916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.110092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eff6e-33d3-4f38-9d19-f7b630805b7b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.158059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.158707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.158990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.159317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.159967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:23.160270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.161670 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.176797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 220/447] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:23.178404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.227711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22cea6-0170-4bc0-84b9-ffceef87de85 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.260707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:23.272354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.273313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.273839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.274123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.293999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:23.318332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040dd73a-e503-482e-8350-96da57591da3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.362240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.362786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.363090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.363414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.363505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.365040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.444162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.445650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.497948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 228/448] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:23.499332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.535562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:23.555461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:23.579039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 221/449] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 18:54:23.585442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.639852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfbac9-e6b4-4735-85c5-56b982fc7693 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.675783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.676526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.676805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.677079 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] PATH_INFO: `/v3/projects/f73f010330ab41eead750a6cbd54038f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.678194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] RBAC: Authorizing `identity:get_project(project_id=f73f010330ab41eead750a6cbd54038f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.679685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.681341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 229/450] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/projects/f73f010330ab41eead750a6cbd54038f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:23.687584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.709992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d123e-90f3-4f44-906e-39b6b807e13a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.744074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.744702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.744977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.745832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] PATH_INFO: `/v3/projects/11b2e035c0894ef7a5efbac9016e862c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.746952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] RBAC: Authorizing `identity:get_project(project_id=11b2e035c0894ef7a5efbac9016e862c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.748413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.750030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 222/451] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/projects/11b2e035c0894ef7a5efbac9016e862c => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:23.755595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.784610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:23.792025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf9ea-425a-40e3-8d8f-fe7d1feadfe7 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.814164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:23.832261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.832904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.833181 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.833455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] PATH_INFO: `/v3/groups/ae519b35941040d3aee900a4f5ca4ad9` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.834642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] RBAC: Authorizing `identity:get_group(group_id=ae519b35941040d3aee900a4f5ca4ad9)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.836115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.840678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 230/452] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/groups/ae519b35941040d3aee900a4f5ca4ad9 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:23.844482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.874906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca434-e784-4138-b950-4fd9dd3ee15c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.914150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.914798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.915165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.915440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] PATH_INFO: `/v3/groups/36ba0cf741f84a1485acbf9ad24d985a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.916554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] RBAC: Authorizing `identity:get_group(group_id=36ba0cf741f84a1485acbf9ad24d985a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.919364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:23.920940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 223/453] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/groups/36ba0cf741f84a1485acbf9ad24d985a => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:23.926683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:23.956558 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc6cd-736c-4a63-ab97-04ddc958be56 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:23.993208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed416694-8631-412f-95bd-2bda842555c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:23.993816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed416694-8631-412f-95bd-2bda842555c7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:23.994095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed416694-8631-412f-95bd-2bda842555c7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:23.994367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed416694-8631-412f-95bd-2bda842555c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:23.995146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed416694-8631-412f-95bd-2bda842555c7 None admin] RBAC: Adding query filter params (group.id=ae519b35941040d3aee900a4f5ca4ad9, role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=f73f010330ab41eead750a6cbd54038f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:23.995928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed416694-8631-412f-95bd-2bda842555c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:23.996973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed416694-8631-412f-95bd-2bda842555c7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:24.003052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 231/454] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/role_assignments?group.id=ae519b35941040d3aee900a4f5ca4ad9&scope.project.id=f73f010330ab41eead750a6cbd54038f&role.id=377956ded99f409684f20afaaf7052c8 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:24.034675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b6496-8768-499b-bd7e-3598abbec364 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:24.057585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:24.074660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81970b77-fc0b-4132-b267-d48fd5c8f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:24.075335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81970b77-fc0b-4132-b267-d48fd5c8f036 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:24.075708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81970b77-fc0b-4132-b267-d48fd5c8f036 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:24.075884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81970b77-fc0b-4132-b267-d48fd5c8f036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:24.076656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81970b77-fc0b-4132-b267-d48fd5c8f036 None admin] RBAC: Adding query filter params (group.id=36ba0cf741f84a1485acbf9ad24d985a, role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=11b2e035c0894ef7a5efbac9016e862c) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:24.076969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81970b77-fc0b-4132-b267-d48fd5c8f036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:24.078470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81970b77-fc0b-4132-b267-d48fd5c8f036 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:24.085199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 224/455] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Aug 29 18:54:23 2022] GET /identity/v3/role_assignments?group.id=36ba0cf741f84a1485acbf9ad24d985a&scope.project.id=11b2e035c0894ef7a5efbac9016e862c&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:26.558195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6eda-c695-4ec3-96cd-19c3846b45d3 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:26.558716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6eda-c695-4ec3-96cd-19c3846b45d3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:26.558716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6eda-c695-4ec3-96cd-19c3846b45d3 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:26.560474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 232/456] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:26.566151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55543b1a-0ab8-468b-bc92-26f421b4ed6d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:26.567043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55543b1a-0ab8-468b-bc92-26f421b4ed6d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:26.567552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55543b1a-0ab8-468b-bc92-26f421b4ed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:26.596061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-55543b1a-0ab8-468b-bc92-26f421b4ed6d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:26.597587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-55543b1a-0ab8-468b-bc92-26f421b4ed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:26.681699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 225/457] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:26.891892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3273cb-16a8-4fc7-8614-3224d90e39f3 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:26.892216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3273cb-16a8-4fc7-8614-3224d90e39f3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:26.892454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3273cb-16a8-4fc7-8614-3224d90e39f3 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:26.894004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 233/458] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:26.899371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98869d-613f-48d2-bb4b-7cf16a63fa0a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:26.899664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98869d-613f-48d2-bb4b-7cf16a63fa0a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:26.899942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98869d-613f-48d2-bb4b-7cf16a63fa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:26.925891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc539ac5-2a67-4277-a961-b979ec030067 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:26.926191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc539ac5-2a67-4277-a961-b979ec030067 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:26.926404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc539ac5-2a67-4277-a961-b979ec030067 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:26.928005 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 234/459] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:26.928971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-2c98869d-613f-48d2-bb4b-7cf16a63fa0a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:26.930173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2c98869d-613f-48d2-bb4b-7cf16a63fa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:26.943003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b50c2c-91da-421a-99cd-2ee8b0be417e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:26.943311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b50c2c-91da-421a-99cd-2ee8b0be417e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:26.943592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b50c2c-91da-421a-99cd-2ee8b0be417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:26.971366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e1b50c2c-91da-421a-99cd-2ee8b0be417e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:26.972563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e1b50c2c-91da-421a-99cd-2ee8b0be417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:27.010364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:27.054925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 226/460] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:27.089488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98869d-613f-48d2-bb4b-7cf16a63fa0a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.099170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 235/461] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:27.159198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:27.218666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98869d-613f-48d2-bb4b-7cf16a63fa0a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:27.254380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:27.255022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.255328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.255596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.267342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:27.268811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:27.284854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:27.285819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:27.286664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:27.286664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:27.286664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:27.286664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 227/462] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:27.292222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b50c2c-91da-421a-99cd-2ee8b0be417e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.443399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b50c2c-91da-421a-99cd-2ee8b0be417e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:27.502269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:27.502901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.503215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.503490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.504364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:27.504529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:27.505346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.505931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:27.520801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 236/463] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:27.521669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.559124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:27.625807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:27.634947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9d0b-3b10-44fe-9723-901a40e7a92c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:27.684380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb843b38-2933-4c34-a1b1-c0e8b8c48efe None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:27.690344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:27.690983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.691291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.691575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.705994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:27.708679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:27.725806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:27.726920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:27.728651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:27.728651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:27.728651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:27.728651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 228/464] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:27.728651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.741766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1d6ef-fbc2-437d-b620-4bf8b02f5a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:27.742991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1d6ef-fbc2-437d-b620-4bf8b02f5a60 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.743502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1d6ef-fbc2-437d-b620-4bf8b02f5a60 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.743968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1d6ef-fbc2-437d-b620-4bf8b02f5a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.756851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1d6ef-fbc2-437d-b620-4bf8b02f5a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:27.758323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1d6ef-fbc2-437d-b620-4bf8b02f5a60 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-04b1d6ef-fbc2-437d-b620-4bf8b02f5a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:27.774498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:27.775726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 237/465] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3ca09-75c6-4594-a824-2b82020b67c4 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3ca09-75c6-4594-a824-2b82020b67c4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3ca09-75c6-4594-a824-2b82020b67c4 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 238/466] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:27.782296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3ca09-75c6-4594-a824-2b82020b67c4 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.858371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8e8a7-2df8-4890-a465-a05539bd3865 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:27.865226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:27.895794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:27.896335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.896619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.896923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.897549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:27.898983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:27.906331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3ca09-75c6-4594-a824-2b82020b67c4 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:27.925905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 229/467] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:54:27 2022] POST /identity/v3/users => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:27.927061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:27.937306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:27.963898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f44abe-9d18-40bf-b040-0d295fd3ad18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:27.964608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f44abe-9d18-40bf-b040-0d295fd3ad18 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.965041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f44abe-9d18-40bf-b040-0d295fd3ad18 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.965893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f44abe-9d18-40bf-b040-0d295fd3ad18 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:27.966778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f44abe-9d18-40bf-b040-0d295fd3ad18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:27.967181 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f44abe-9d18-40bf-b040-0d295fd3ad18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:27.968785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f44abe-9d18-40bf-b040-0d295fd3ad18 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:27.987565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 239/468] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:27.989414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b48f-e20f-443e-a326-9c7a2e43160b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:27.989966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b48f-e20f-443e-a326-9c7a2e43160b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:27.990479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b48f-e20f-443e-a326-9c7a2e43160b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.017756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d498b48f-e20f-443e-a326-9c7a2e43160b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:28.019475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d498b48f-e20f-443e-a326-9c7a2e43160b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:28.081508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a2dd-b60b-4071-aaa1-dedd5b66c387 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:28.105297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 240/469] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:28.105836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498b48f-e20f-443e-a326-9c7a2e43160b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:28.131195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:28.131816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.132102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.132382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.133040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:28.133349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:28.135318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:28.155338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 230/470] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:28.165985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:28.185045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:28.234375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498b48f-e20f-443e-a326-9c7a2e43160b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:28.268213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:28.276002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af7a0-d334-49f2-a8c9-111575234ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:28.276969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af7a0-d334-49f2-a8c9-111575234ea3 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.277255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af7a0-d334-49f2-a8c9-111575234ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.277530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af7a0-d334-49f2-a8c9-111575234ea3 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.278354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9af7a0-d334-49f2-a8c9-111575234ea3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:28.280210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9af7a0-d334-49f2-a8c9-111575234ea3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:28.294058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33819321-6f42-4e3f-9724-3c4847528f16 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:28.306750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 241/471] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:54:28 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:28.309472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5ea6-f8e0-4b9e-8f4c-385cc9f75e7b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.309766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5ea6-f8e0-4b9e-8f4c-385cc9f75e7b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.310047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5ea6-f8e0-4b9e-8f4c-385cc9f75e7b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.312343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 242/472] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:28.317674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365585fb-de96-4935-b702-2f725e29a8e2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.317977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365585fb-de96-4935-b702-2f725e29a8e2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.318491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365585fb-de96-4935-b702-2f725e29a8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.344320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:28.344992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.345287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-365585fb-de96-4935-b702-2f725e29a8e2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:28.345398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.345644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.346318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:28.347453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-365585fb-de96-4935-b702-2f725e29a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:28.347897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:28.402124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 231/473] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:54:28 2022] POST /identity/v3/users => generated 248 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:54:28.420654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:28.456150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 243/474] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:28.607046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68702a64-6774-40a7-b898-c92d68b26571 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:28.631181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:28.649522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:28.650164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.650450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.650727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.664840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:28.666403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:28.681951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:28.683222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:28.684052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:28.684052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:28.684052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:28.684052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 232/475] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:28.689543 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365585fb-de96-4935-b702-2f725e29a8e2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:28.743783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:28.791863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:28.808848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365585fb-de96-4935-b702-2f725e29a8e2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:28.859692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:28.860231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:28.860509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:28.860783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:28.864539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:28.864871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:28.866267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:28.893850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 244/476] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:28.908885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:28.999103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e1831-37cd-4f52-82a4-22496e2b72b7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:29.042865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:29.043559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:29.043842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:29.044120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:29.044815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:29.046187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:29.054581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.sql.core [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:54:29.054581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 18:54:29.054581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [parameters: {'id': '3a6d42f9be66400681aea2f8cbab5248', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 18:54:29.054581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=76732) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application result.read() Aug 29 18:54:29.071741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:54:29.072641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application yield resource Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise self.value Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application session.commit() Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:54:29.073562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:54:29.074473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:54:29.075394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application result.read() Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:54:29.076397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [parameters: {'id': '3a6d42f9be66400681aea2f8cbab5248', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:29.077169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 18:54:29.077944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:29.078755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 233/477] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Aug 29 18:54:28 2022] POST /identity/v3/roles => generated 144 bytes in 282 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:29.080160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:29.120407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afe557-7fb0-4569-ba77-0f84a1ee010c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:29.139105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:29.145578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:29.199891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:29.200528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:29.200914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:29.201195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:29.201860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:29.207981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:29.249018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 245/478] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:54:28 2022] POST /identity/v3/users => generated 249 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:54:29.269357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8dbae-83f4-4d5a-9316-44f78dabe305 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:29.310404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646206a4-fd8f-476d-9505-e07a64c36a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:29.311166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646206a4-fd8f-476d-9505-e07a64c36a73 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:29.311456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646206a4-fd8f-476d-9505-e07a64c36a73 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:29.311731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646206a4-fd8f-476d-9505-e07a64c36a73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:29.323764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646206a4-fd8f-476d-9505-e07a64c36a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-646206a4-fd8f-476d-9505-e07a64c36a73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:29.328754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:29.329630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:29.330421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 234/479] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:54:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:29.337474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:29.486618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476014ed-ff8c-4d52-b2ea-da3160a5037f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:29.524095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b03bf2-2f84-4252-a45d-8fea48478c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:29.524764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b03bf2-2f84-4252-a45d-8fea48478c73 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:29.525046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b03bf2-2f84-4252-a45d-8fea48478c73 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:29.525373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b03bf2-2f84-4252-a45d-8fea48478c73 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:29.526043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b03bf2-2f84-4252-a45d-8fea48478c73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:29.526398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b03bf2-2f84-4252-a45d-8fea48478c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:29.528522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b03bf2-2f84-4252-a45d-8fea48478c73 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:29.538271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 246/480] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:30.389276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e06fd-ac78-497e-9146-7b7f7a05f01d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:30.389806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e06fd-ac78-497e-9146-7b7f7a05f01d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:30.390102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e06fd-ac78-497e-9146-7b7f7a05f01d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:30.391609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 235/481] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:30.397193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753573d7-a10a-44f6-a9b2-40c5894dcbaf None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:30.397663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753573d7-a10a-44f6-a9b2-40c5894dcbaf None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:30.397740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753573d7-a10a-44f6-a9b2-40c5894dcbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:30.424624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-753573d7-a10a-44f6-a9b2-40c5894dcbaf None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:30.425827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-753573d7-a10a-44f6-a9b2-40c5894dcbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:30.450791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:30.516977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 247/482] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:30.847881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e06fd-ac78-497e-9146-7b7f7a05f01d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:30.956841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9496d-547c-40f2-bbac-5ea86fc9f393 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:30.957188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9496d-547c-40f2-bbac-5ea86fc9f393 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:30.957469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9496d-547c-40f2-bbac-5ea86fc9f393 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:30.958799 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 248/483] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:30.964654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826a0d-b9ab-4e87-aafa-b2ff063cee3e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:30.965199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826a0d-b9ab-4e87-aafa-b2ff063cee3e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:30.965708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826a0d-b9ab-4e87-aafa-b2ff063cee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:30.979176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:30.990857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e06fd-ac78-497e-9146-7b7f7a05f01d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.007611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a2826a0d-b9ab-4e87-aafa-b2ff063cee3e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:31.009249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a2826a0d-b9ab-4e87-aafa-b2ff063cee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:31.032297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.032926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.033202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.033466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.044979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:31.050172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:31.051062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:31.051880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 236/484] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:31.061025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:31.151406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 249/485] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:31.153481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457ed0-f595-472c-9aa0-9cc14031da23 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.153766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457ed0-f595-472c-9aa0-9cc14031da23 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.154036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457ed0-f595-472c-9aa0-9cc14031da23 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.160017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 250/486] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:31.165184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8a4d2-f01d-4363-81b5-06ea40b1b1ec None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.165464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8a4d2-f01d-4363-81b5-06ea40b1b1ec None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.165734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8a4d2-f01d-4363-81b5-06ea40b1b1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.196814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfb530-602d-4477-9619-e64aff6c91be None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.202059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9dd8a4d2-f01d-4363-81b5-06ea40b1b1ec None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:31.203282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9dd8a4d2-f01d-4363-81b5-06ea40b1b1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:31.240607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.241216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.241489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.241754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.242423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:31.242720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:31.244310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:31.264140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 237/487] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:31.269590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:31.306749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:31.351846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 251/488] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:31.413358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552cdc-f91a-463c-a41b-38b696eeee9d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.450992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.451661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.451938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.452215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.463800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:31.466334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:31.483229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:31.485134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:31.486260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:31.486260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:31.486260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:31.486260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 238/489] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:31.491939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8a4d2-f01d-4363-81b5-06ea40b1b1ec None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:31.523768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:31.545721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:31.643016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:31.678769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c7783-9098-41ad-8f9e-7a88ffaa021c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.718834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.719920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.720211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.720758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.726227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8a4d2-f01d-4363-81b5-06ea40b1b1ec None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.736005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:31.742911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:31.743841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:31.744647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 239/490] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:31.751109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:31.776265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.776875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.777239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.777676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.778846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:31.779492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:31.781157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:31.797261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 252/491] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:31.798116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:31.876137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dcb8-6d21-4f83-8420-43b58d071077 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.895537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:31.911984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:31.920626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.921406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.921690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.921962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.934343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:31.935293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde6bbc-5b3c-4a9e-a33a-c0a216f2fc72 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:31.939857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:31.940753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:31.941546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:31.945811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 240/492] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:31.946840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:31.977026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:31.977794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:31.978068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:31.978666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:31.980246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:31.980557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:31.982237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:31.995192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 253/493] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:31.996716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.069762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9a1e5-91e3-47ab-b855-b1f30bc54d02 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.103417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfd2c1-23bc-4f9a-acdf-c32b3c24964c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.108803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.109599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.110018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.110418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.119305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.120963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:32.134942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:32.136121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:32.137270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 241/494] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:32.138549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.149110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e99ca-42b3-462a-a837-eeb6b441c5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.149724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e99ca-42b3-462a-a837-eeb6b441c5d1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.149995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e99ca-42b3-462a-a837-eeb6b441c5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.150271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e99ca-42b3-462a-a837-eeb6b441c5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.150933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e99ca-42b3-462a-a837-eeb6b441c5d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:32.151903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e99ca-42b3-462a-a837-eeb6b441c5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.153751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e99ca-42b3-462a-a837-eeb6b441c5d1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.155266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:32.163329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 254/495] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:32.164564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe756e-df45-4d53-a9b2-c32ce7d0a18a None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.164915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe756e-df45-4d53-a9b2-c32ce7d0a18a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.165194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe756e-df45-4d53-a9b2-c32ce7d0a18a None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.169741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 255/496] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:32.169741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe756e-df45-4d53-a9b2-c32ce7d0a18a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.213320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:32.268760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96025-a2b4-4653-acb8-a1709019d9ca None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.277795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe756e-df45-4d53-a9b2-c32ce7d0a18a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.304535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc73155-096c-472d-ad25-24bb79f1bf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.305162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc73155-096c-472d-ad25-24bb79f1bf12 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.305446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc73155-096c-472d-ad25-24bb79f1bf12 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.305727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc73155-096c-472d-ad25-24bb79f1bf12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.313334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cab72-d559-441d-b4a5-3a03b5676c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.313939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cab72-d559-441d-b4a5-3a03b5676c4b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.314214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cab72-d559-441d-b4a5-3a03b5676c4b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.314481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cab72-d559-441d-b4a5-3a03b5676c4b None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.315242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cab72-d559-441d-b4a5-3a03b5676c4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:32.315544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cab72-d559-441d-b4a5-3a03b5676c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.317062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cab72-d559-441d-b4a5-3a03b5676c4b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.318324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc73155-096c-472d-ad25-24bb79f1bf12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.319756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc73155-096c-472d-ad25-24bb79f1bf12 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-5bc73155-096c-472d-ad25-24bb79f1bf12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:32.342120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:32.343093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:32.343966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:32.343966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:32.343966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:32.343966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 242/497] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:32.345253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a8f6-c52f-444f-ae5f-0cb88056a0de None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.345253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a8f6-c52f-444f-ae5f-0cb88056a0de None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.345253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a8f6-c52f-444f-ae5f-0cb88056a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.357588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 256/498] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:32.358854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f03d41-ea40-4224-a325-1995d462792d None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.359166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f03d41-ea40-4224-a325-1995d462792d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.359445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f03d41-ea40-4224-a325-1995d462792d None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.360858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 257/499] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:32.365348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f03d41-ea40-4224-a325-1995d462792d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.365464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cba7a8f6-c52f-444f-ae5f-0cb88056a0de None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:32.366538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cba7a8f6-c52f-444f-ae5f-0cb88056a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:32.428093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:32.438254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 243/500] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:32.439152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7a8f6-c52f-444f-ae5f-0cb88056a0de None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.473308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:32.478884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f03d41-ea40-4224-a325-1995d462792d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.520190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.521569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.521849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.522121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.533360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.534817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.550203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7a8f6-c52f-444f-ae5f-0cb88056a0de None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:32.552058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:32.552921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:32.559231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:32.559231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:32.559231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:32.559231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 258/501] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:32.559231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.597839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc9a05-991c-4899-9a70-5b8a57bcc0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.598463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc9a05-991c-4899-9a70-5b8a57bcc0d5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.598744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc9a05-991c-4899-9a70-5b8a57bcc0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.599019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc9a05-991c-4899-9a70-5b8a57bcc0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.599706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc9a05-991c-4899-9a70-5b8a57bcc0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:32.600498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc9a05-991c-4899-9a70-5b8a57bcc0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.601895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc9a05-991c-4899-9a70-5b8a57bcc0d5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.616063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 244/502] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:32.617193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21971caa-43b0-4aee-9b30-a3d77f3c8752 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.617486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21971caa-43b0-4aee-9b30-a3d77f3c8752 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.617877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21971caa-43b0-4aee-9b30-a3d77f3c8752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.637818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-21971caa-43b0-4aee-9b30-a3d77f3c8752 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:32.639010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-21971caa-43b0-4aee-9b30-a3d77f3c8752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:32.690418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0eca06-a56e-4b31-9236-cd5651d0d5d9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.699408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:32.728948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.729535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.729823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.730098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.731706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 245/503] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:32.732672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21971caa-43b0-4aee-9b30-a3d77f3c8752 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.759284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:32.808815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.810400 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.851403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21971caa-43b0-4aee-9b30-a3d77f3c8752 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:32.890136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:32.890752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:32.891034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:32.891338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:32.891992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:32.892302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:32.893666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:32.909676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 246/504] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:32.909676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.934416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 259/505] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:32.934416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:32.941745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.004781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.029031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae71dc-24ee-4c61-975c-e1a936517ff8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.061290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb5406-0c7c-4dbb-9331-5cd98a2d45e3 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.066668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.067342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.067705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.067903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.075888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.077334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:33.089352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:33.090291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:33.091426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 247/506] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:33.092673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.097747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.098377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.098655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.098923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.110277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:33.115563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:33.116660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 260/507] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:33.117736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.189311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.198794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091097-d712-48fe-a24d-c55be3af6023 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.224769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491872e-52d5-43de-8f7f-44bb69be26db None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.235338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.235985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.236263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.236539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] PATH_INFO: `/v3/users/glance` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.245177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.246834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:33.259716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:33.260662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:33.261828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 248/508] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:33.262766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.264500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.267677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.268316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.268597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.268983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.280053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.281536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:33.297060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:33.298066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:33.298902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:33.298902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:33.298902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:33.298902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 261/509] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:33.298902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.375261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc379-1d97-46d8-bb55-7725ab110567 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.401136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3949ab-e8b6-493d-8e7d-16f454343ca6 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.409904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.410539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.410820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.411130 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.420342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.422668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.424188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.437968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:33.438554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:33.439849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:33.440709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:33.440709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:33.440709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:33.440870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.440870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.440870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.440870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:33.441119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 249/510] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:33.441192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.441266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.441834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.461354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 262/511] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:33.462101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.507645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.556720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b1f6-32ee-4cb6-8586-0abd7ef3c9c9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.570191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151da1f-82c1-4bf1-a92f-bd54341e7700 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.591383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.591993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.592270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.592544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.593206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:33.593508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.594877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.602897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 250/512] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:33.603719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.605359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.605991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.606279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.606554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.607399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:33.608274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.609198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.626912 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 263/513] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:33.627804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.668875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.714851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876ea6-9e2c-465e-ad72-781e1f39147f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.731245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.737036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e1f93-2235-4c9b-9eec-92ad2531961c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.750869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.751542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.751821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.752106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.752773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:33.753087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.754513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.769557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 251/514] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:33.770438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.775828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.777493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.777773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.778050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.778710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:33.779019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.780837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.793827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 264/515] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:33.794951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:33.885159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee5c3-9005-4062-95bd-0b32d7edec74 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.906070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.913899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4189ff-9dd1-4613-b893-a810f96ba287 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:33.922789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.923440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.923716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.923990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.949868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:33.950632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:33.950912 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:33.951237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:33.962522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:33.964731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:33.970313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:33.980637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:33.981579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:33.982445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:33.982445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:33.982445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:33.982445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 265/516] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:33.982654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.000386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.001854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.087424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25d57-e8f2-4911-b7a9-f918aed35869 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.108853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 252/517] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:34.109708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.122111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.122816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.123109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.123383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.140294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:34.196284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:34.215320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.216857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.224794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a00ed9-23f8-448e-b96e-cb1bc0550d5b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.264565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.265199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.265481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.265756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.272497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.273942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:34.286865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:34.287934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:34.288798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 253/518] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:34.289263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.351730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 266/519] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:34.352503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.400040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ecfb2-eea2-4fbe-927b-96a6982c1f98 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.406560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:34.435819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.436393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.436693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.436963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.437599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.439039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.447936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:34.462609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 254/520] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:54:34 2022] POST /identity/v3/users => generated 247 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:34.463263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.475270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2409ee-694a-43f7-9344-e2ffe1a05e4e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.516852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.517583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.517934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.518285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.519016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:34.519450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.520934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.537767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 267/521] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:34.539486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.600394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bad0d-5b58-4b1a-9f13-95e11aca71b7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.643722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.644635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.645055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.645479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.658897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.660403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.672862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f7571-bb64-4f9b-8c57-dccf7772f79a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:34.678832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:34.679817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:34.680681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:34.680681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:34.680681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:34.680681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 255/522] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:34.683275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.688131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:34.719860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.720667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.720783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.720985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.721704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:34.722008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.723520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.736995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:34.744318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 268/523] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:34.750761 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.798072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b779-64ba-401d-90e0-d6d1d75902cc None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.840471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.841113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.841392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.841665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.858634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.860145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.877416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d67478-8a94-43cd-8ed0-b11cb9fd13ab None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:34.877909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:34.878978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:34.879997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:34.879997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:34.879997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:34.879997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 256/524] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:34.880222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:34.924677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:34.925315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:34.925766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:34.926215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:34.939093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:34.941656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:34.963242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:34.964687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 269/525] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:34.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.013106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.013605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.028650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc1beb-898e-408e-8f45-7e432ca5c643 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.067204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.067818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.068097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.068373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.069021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:35.069329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.070704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.084740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 257/526] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:35.085657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.109549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3730e-9632-4fcf-94eb-e5322b911ee8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.151940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.152843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.153126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.153401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.154297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=8adead3860f144099efa11cefa83a37f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:35.154935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.157242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.175243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 270/527] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:34 2022] GET /identity/v3/role_assignments?user.id=8adead3860f144099efa11cefa83a37f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:35.175243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.202300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30204063-f1da-4535-a883-861fa92dfeb8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.245579 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.246363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.246647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.246922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.247620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:35.247928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.249433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.264246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 258/528] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:35.265199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.287725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.304635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.317381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b44766-c4a4-48cd-978e-691ea727bb02 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.358302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.358949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.359259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.359532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.360264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:35.360568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.362070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.389732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 271/529] 10.222.0.94 () {56 vars in 1139 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:35.389732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.397970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328629d-e25b-46bd-8267-05c5c162d1b1 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.441200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.441845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.442023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.442365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.450385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.452278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:35.465050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:35.466276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 259/530] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:35.467169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.534414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e1e25-f20f-4653-b431-a134a4c9da7e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.561117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.587722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f3051-556e-4f9c-8a43-d6de2476a319 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.587940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.588421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.589244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.589516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.598638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.600656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:35.618675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:35.619735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:35.620622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 272/531] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:35.627773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.640862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.641205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.642589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.642897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.643324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.746811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.748702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.749041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be743cb-7181-4abb-b618-2ba8210ed118 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.801724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.805335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.805628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.805909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.806879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:35.807228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.809092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.826130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 273/532] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:35.831516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.846542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.875433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 260/533] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:35.876677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:35.912130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:35.951273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f584d-80d8-4e17-931a-15d5c49d02ac None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:35.989507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:35.990120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:35.990404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:35.990680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:35.991413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:35.991751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:35.993458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:35.996496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f52149-5ff5-462f-b1ed-0f3a45fc9f96 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:36.007777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 274/534] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:36.008641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:36.037227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:36.038225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.038738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.039227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.040259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=ee56cf453075431ab2791e15e803d51c) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:36.040760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:36.043454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:36.055395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 261/535] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:35 2022] GET /identity/v3/role_assignments?user.id=ee56cf453075431ab2791e15e803d51c&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:36.056674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:36.131329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6b048-0736-4f5d-845e-342d22caecdc None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:36.137291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:36.184940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:36.208300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54447bc3-ad93-43d4-84a2-3cb82126effa None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:36.210702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b5f5-f189-48e3-b9ef-1bbd0b977111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:36.211557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b5f5-f189-48e3-b9ef-1bbd0b977111 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.211991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b5f5-f189-48e3-b9ef-1bbd0b977111 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.212434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b5f5-f189-48e3-b9ef-1bbd0b977111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.237746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b5f5-f189-48e3-b9ef-1bbd0b977111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:36.239465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b5f5-f189-48e3-b9ef-1bbd0b977111 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:36.262513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:36.264417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.264705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.264981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.265735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=1d9d979b6539460e82759320c3d96a79) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:36.266046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:36.267529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2e63b5f5-f189-48e3-b9ef-1bbd0b977111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:36.271858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:36.272723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:36.273647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:36.273647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:36.273647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:36.281358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 262/536] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:36 2022] GET /identity/v3/role_assignments?user.id=1d9d979b6539460e82759320c3d96a79&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:36.281549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 275/537] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:36.283125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:36.453582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de0fd2-8c54-40e9-95f3-14973f8d35a8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:36.456302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:36.496299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:36.497390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.497687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.497959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.498610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:36.498911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:36.500306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:36.505769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f5d5e-77af-4049-91b7-2cbb03e30835 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.506056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f5d5e-77af-4049-91b7-2cbb03e30835 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.506332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f5d5e-77af-4049-91b7-2cbb03e30835 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.508138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 276/538] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:36.514879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff690de-6f6d-48f5-961b-488b95435868 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.515047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 263/539] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:36.515162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff690de-6f6d-48f5-961b-488b95435868 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.515389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff690de-6f6d-48f5-961b-488b95435868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.520446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:36.542544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-eff690de-6f6d-48f5-961b-488b95435868 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:36.544093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-eff690de-6f6d-48f5-961b-488b95435868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:36.674612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 277/540] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:36.691796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a8bd0-57b1-478c-884c-5a55e5014f1f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:36.728785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:36.729942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.730212 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.730516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.738221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:36.739742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:36.752166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:36.754007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:36.755143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 264/541] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:36.762398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff690de-6f6d-48f5-961b-488b95435868 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:36.905883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff690de-6f6d-48f5-961b-488b95435868 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:36.920208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:36.952780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:36.953414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:36.953688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:36.953959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:36.954746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:36.955054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:36.956637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:36.978247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 278/542] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:36.985636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:37.015560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:37.051467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:37.144901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488b9ca-41dc-45d6-8407-77c4274f2f96 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:37.197359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:37.197996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.198838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.198985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c03e-9494-4093-9cdf-18d7b3011515 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:37.199175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.199864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=70c6682b92694484b2fa2710314f547f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:37.200168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:37.202079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:37.208368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 265/543] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:36 2022] GET /identity/v3/role_assignments?user.id=70c6682b92694484b2fa2710314f547f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:37.243021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:37.243688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.243973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.244249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.257318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:37.263563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:37.264643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:37.266216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 279/544] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:37.270891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:37.422456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:37.437607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442b367-05db-44bb-95ea-93ba33037872 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:37.474763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c3e39-9ec5-4c31-9d0e-3704e57f0890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:37.475553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c3e39-9ec5-4c31-9d0e-3704e57f0890 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.475736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c3e39-9ec5-4c31-9d0e-3704e57f0890 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.475973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c3e39-9ec5-4c31-9d0e-3704e57f0890 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.476614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c3e39-9ec5-4c31-9d0e-3704e57f0890 None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:37.476923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c3e39-9ec5-4c31-9d0e-3704e57f0890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:37.478296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c3e39-9ec5-4c31-9d0e-3704e57f0890 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:37.491188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 266/545] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:37.493143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:37.521702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3fba-5b05-40d4-bce4-c4301787c735 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.522579 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3fba-5b05-40d4-bce4-c4301787c735 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.523517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3fba-5b05-40d4-bce4-c4301787c735 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.527496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 267/546] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:54:37.534027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7246a-21e2-46f9-8683-d35a436fbadf None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.534256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:37.535051 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7246a-21e2-46f9-8683-d35a436fbadf None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.535826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7246a-21e2-46f9-8683-d35a436fbadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.580138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-60e7246a-21e2-46f9-8683-d35a436fbadf None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:37.581369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-60e7246a-21e2-46f9-8683-d35a436fbadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:37.631964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6c652-3024-4cdc-b65c-3293f98f70ac None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:37.682094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 268/547] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:37.686438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:37.687101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.687398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.687674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.704008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:37.705826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:37.732918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:37.733879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:37.734695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:37.734695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:37.734695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:37.736837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 280/548] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:37.743336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7246a-21e2-46f9-8683-d35a436fbadf None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:37.869071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:37.882543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7246a-21e2-46f9-8683-d35a436fbadf None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:37.920714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:37.921346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:37.921630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:37.921906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:37.922565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:37.922872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:37.924291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:37.938132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 269/549] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:37.944720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.041408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.065019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5b528-f490-4426-9505-e22a416f312b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.111987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:38.112633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.112912 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.113183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] PATH_INFO: `/v3/users/swift` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.123638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:38.124948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.125490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:38.140439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:38.141614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:38.143972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 281/550] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:38.152024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.283290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a152823-e606-4ae9-8f4d-8d0585c49baa None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.326319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:38.327321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.327825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.328412 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.329421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:38.329929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:38.331773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:38.352037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.365705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 282/551] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Aug 29 18:54:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:38.372436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.407280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8beb1-9b9f-4cf4-bc86-f15965aadaab None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.467437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:38.468315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.468768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.469339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.478754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.483277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:38.490429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:38.491801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:38.492810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:38.499127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 270/552] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:38.499501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3badf72-268a-4f04-aa4c-7fce0ac2cb44 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.515386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.539692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:38.540791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.541293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.542328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.618265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.620944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:38.622687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:38.656144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f48d6-5375-460e-8c23-65a20826e388 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.694962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd8416-ce86-4566-b88d-34ad159fe28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:38.695606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd8416-ce86-4566-b88d-34ad159fe28e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.695944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd8416-ce86-4566-b88d-34ad159fe28e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.696219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd8416-ce86-4566-b88d-34ad159fe28e None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.696882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd8416-ce86-4566-b88d-34ad159fe28e None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:38.697281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd8416-ce86-4566-b88d-34ad159fe28e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:38.698964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd8416-ce86-4566-b88d-34ad159fe28e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:38.708277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 271/553] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:38.709549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc83e81-dc65-413d-b13d-988bac05f7c7 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.709828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc83e81-dc65-413d-b13d-988bac05f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.710097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc83e81-dc65-413d-b13d-988bac05f7c7 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.711580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 272/554] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:38.718081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc83c6-cce8-43c2-a51b-b9cf9eab5ada None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.718655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc83c6-cce8-43c2-a51b-b9cf9eab5ada None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.719468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc83c6-cce8-43c2-a51b-b9cf9eab5ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.739977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c6dc83c6-cce8-43c2-a51b-b9cf9eab5ada None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:38.741190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c6dc83c6-cce8-43c2-a51b-b9cf9eab5ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:38.742242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 283/555] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:38.748253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.812279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.824974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 273/556] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:38.827245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af6dd1-6cf0-4dd7-951c-7af4516cfe72 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.827791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af6dd1-6cf0-4dd7-951c-7af4516cfe72 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.828510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af6dd1-6cf0-4dd7-951c-7af4516cfe72 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.831368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 274/557] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:38.832459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af6dd1-6cf0-4dd7-951c-7af4516cfe72 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:38.883527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:38.888089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a020d-f2ae-46f2-a1e6-f15ef9c8f62e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.931186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fe9d4-507f-44f6-84a8-f220297b4a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:38.932173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fe9d4-507f-44f6-84a8-f220297b4a78 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.933273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fe9d4-507f-44f6-84a8-f220297b4a78 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.933787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fe9d4-507f-44f6-84a8-f220297b4a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.944856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fe9d4-507f-44f6-84a8-f220297b4a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:38.946952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fe9d4-507f-44f6-84a8-f220297b4a78 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-552fe9d4-507f-44f6-84a8-f220297b4a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:38.969116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:38.971225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:38.972355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:38.972355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:38.972355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:38.975232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 284/558] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:38.977452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6849cd-7c54-4bfc-9bf6-ccec2792b9bb None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:38.977727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6849cd-7c54-4bfc-9bf6-ccec2792b9bb None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:38.978081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6849cd-7c54-4bfc-9bf6-ccec2792b9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:38.978845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af6dd1-6cf0-4dd7-951c-7af4516cfe72 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:38.999463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2d6849cd-7c54-4bfc-9bf6-ccec2792b9bb None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:39.001110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2d6849cd-7c54-4bfc-9bf6-ccec2792b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:39.025923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.026602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.027128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.027282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.039512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.041386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:39.058224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:39.059563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:39.060785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:39.060785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:39.060785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:39.060785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 275/559] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:39.064823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.082145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 285/560] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:39.088129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6849cd-7c54-4bfc-9bf6-ccec2792b9bb None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.117373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.185266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d369d-1c28-4133-b98c-fe926d6f36a8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.211525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.222923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.224047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.224400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.224743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.225464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:39.225834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.227282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.239272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6849cd-7c54-4bfc-9bf6-ccec2792b9bb None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.248558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 276/561] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:39.248558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.291324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.293064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.293507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.293792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.294579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:39.294890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.296390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.314668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 286/562] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:39.314668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.370273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13eb37-545a-49af-8e73-7e32dc80dcfc None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.373780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.411098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.411983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.412359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.412731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.425085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:39.430202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:39.431172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:39.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:39.432728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d627f91-5035-44f5-a70a-be04993e7eac None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.432803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 277/563] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:39.432803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.471973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.472607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.472885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.473161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] PATH_INFO: `/v3/users/nova` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.481640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.483122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:39.495633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:39.496723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 287/564] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:39.497810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.501503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.549015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b1744-1529-4ec9-a18f-d6d0fb3da384 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.586909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.587581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.587859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.588130 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.598408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.599970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:39.613015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:39.613945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:39.614843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 278/565] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:39.615795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.627190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7459a2-17aa-400d-9ded-9d564b632acd None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.643228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.663880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.664502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.664777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.665047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.675749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:39.680978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:39.682087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 288/566] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:39.683371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.725553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6370b-d002-4b83-a605-e7ca19d5fde9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.747763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.760103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.761243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.761516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.762036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.762696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:39.762997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.764420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.772385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 279/567] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:39.773182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.787445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4404d0-a31d-4067-be0d-0a23f06fb44a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.822223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7429e4-6cd8-41e0-ba50-1576f43a1227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.822832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7429e4-6cd8-41e0-ba50-1576f43a1227 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.823231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7429e4-6cd8-41e0-ba50-1576f43a1227 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.823603 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7429e4-6cd8-41e0-ba50-1576f43a1227 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.824329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7429e4-6cd8-41e0-ba50-1576f43a1227 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:39.824635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7429e4-6cd8-41e0-ba50-1576f43a1227 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.826070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7429e4-6cd8-41e0-ba50-1576f43a1227 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.843678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 289/568] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:39.844747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c7e08-28cc-45a1-aa47-39522242df8b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.845031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c7e08-28cc-45a1-aa47-39522242df8b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.845301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c7e08-28cc-45a1-aa47-39522242df8b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.850559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 290/569] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:39.850559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c7e08-28cc-45a1-aa47-39522242df8b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.865139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.884693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32307b-cd94-4f28-8c69-200331f29671 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.919592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.920193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.920469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.920740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.921448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:39.921750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.923182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:39.941618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 280/570] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:39.942450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:39.956532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c7e08-28cc-45a1-aa47-39522242df8b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:39.977117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:39.992830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edaaa09-caf0-4a4b-9ae7-6b9ee953d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:39.993443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaaa09-caf0-4a4b-9ae7-6b9ee953d1ae None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:39.993716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaaa09-caf0-4a4b-9ae7-6b9ee953d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:39.993988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaaa09-caf0-4a4b-9ae7-6b9ee953d1ae None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:39.994649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaaa09-caf0-4a4b-9ae7-6b9ee953d1ae None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:39.995595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaaa09-caf0-4a4b-9ae7-6b9ee953d1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:39.997111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaaa09-caf0-4a4b-9ae7-6b9ee953d1ae None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.005738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 291/571] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:40.007287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857a2f-c3c7-4535-aca5-ba1ef0265bd2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.007571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857a2f-c3c7-4535-aca5-ba1ef0265bd2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.007891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857a2f-c3c7-4535-aca5-ba1ef0265bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.028468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-83857a2f-c3c7-4535-aca5-ba1ef0265bd2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:40.029667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-83857a2f-c3c7-4535-aca5-ba1ef0265bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:40.050609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb4b3-5ef2-48e0-aa6c-2209e1a2e1a0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.086171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.086815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.087118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.087396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.097840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 292/572] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:40.098007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.098674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83857a2f-c3c7-4535-aca5-ba1ef0265bd2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.099348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.107713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:40.117462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:40.118509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:40.124715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:40.124715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:40.124715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:40.124715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 281/573] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:40.124715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.225576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83857a2f-c3c7-4535-aca5-ba1ef0265bd2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.241599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e91a12-dd63-4fd2-8df3-71f2baa9fd2e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.242341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.272594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.273231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.273507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.273772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.280496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.281105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.281390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.281665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.282403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=7a0e57abdb8d4966a2066a85bbfbc15e) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:40.283206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.285067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.286506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.287968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.302256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 282/574] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/role_assignments?user.id=7a0e57abdb8d4966a2066a85bbfbc15e&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:40.302256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:40.314432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:40.315638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:40.316571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:40.316571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:40.316571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:40.318433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 293/575] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:40.319378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.393403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.441047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456de07-ce21-4d3d-9cd0-7e93df74c7d8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.445574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268d0b4-5737-4330-be03-33ab8bf7d645 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.484038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.484575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.484873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.485127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.485767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:40.486072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.487493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.490529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.491186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.491466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.491734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.500020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 294/576] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:40.504527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.505485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.505990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:40.524862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:40.525873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:40.526839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:40.526839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:40.526839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:40.526839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 283/577] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:40.528151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.543780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.628419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fdada-ca23-43c5-81a9-7a576d1ecad7 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.649776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef52f2-73b7-43e5-a88d-59589fe964fa None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.665422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.670477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.671131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.671451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.671730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.672387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:40.672695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.674548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.687404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 295/578] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:40.688256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.695270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.695926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.696207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.696479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.708978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:40.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:40.715940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:40.716785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 284/579] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:40.717493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.790531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:40.808089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150e609-b1cf-4fba-bddf-85bfbb2c81bf None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.838270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e5fe-5013-437e-8014-6c2529096bc7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:40.855748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.856808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.857292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.857779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] PATH_INFO: `/v3/users/glance` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.872253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.874921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.879199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:40.879806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:40.880091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:40.880366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:40.881078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:40.881319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:40.882664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:40.900609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 285/580] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:40.900609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:40.900863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:40.902103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:54:40.903163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:40.904600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 296/581] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:40.905333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:40.966151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.031018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548bf18-1a8a-472f-b5aa-e0678fa5af37 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.047608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c959-f85f-4557-a3c2-efd561a840fd None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.069470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.070467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.071259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.071637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.072319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:41.072625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.076121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.094144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 297/582] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:41.095014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.105884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.106852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.107673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.108453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.109175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.118926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.121408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:41.134559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:41.136245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:41.138184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 286/583] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:41.138901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.236836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4a5f-144f-48de-9677-60f777f5b43d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.246630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.255383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a07e28-3895-48f2-88de-c09f152199e0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.273016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.273735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.273957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.274237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.283398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.284891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.296710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.297348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.297707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.297954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:41.298109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:41.308347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:54:41.310262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:41.312344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:41.312344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.312809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 298/584] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:41.312809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.313261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.349622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 287/585] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:41.350481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.401924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.465605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca56d-cfe7-4f38-a8ca-10467e072ce0 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.477644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad25b1-c59a-4ac5-b57c-b2f5f2f02f7a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.509727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.510372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.510871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.510999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.523292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.524984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.526772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.527432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.527712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.527993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.528750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:41.530257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.531538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:41.543240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:41.544446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:41.545467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:41.545467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:41.545467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:41.545467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 299/586] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:41.545809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.548920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 288/587] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:41.550074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.567919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.669151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc61b90-9b14-426e-8a8d-d52599d7739d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.676574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6324f6-2ebc-491a-a35e-28f759093d86 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.689351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.709444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.710051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.710330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.710600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.711358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:41.711662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.713097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.717857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.718779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.719306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.719763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.732580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.733215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 300/588] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:41.734079 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.734736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:41.756876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:41.758066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:41.759267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:41.759267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:41.759267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:41.763453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 289/589] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:41.764325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.841112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.857842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25c273-1d4b-4e53-b4d9-ea39eeaf6572 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.895195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f54b00-d052-47f1-a23c-f5430dded19d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:41.900110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.900833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.901296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.901774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.902553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:41.902927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.904436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.916959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 301/590] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:41.918643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:41.933952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:41.935268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:41.935725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:41.936270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/8adead3860f144099efa11cefa83a37f/roles/6328cebc97c34f12bc2c15bafa4ba1e0` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:41.953191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=8adead3860f144099efa11cefa83a37f, role_id=6328cebc97c34f12bc2c15bafa4ba1e0)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:41.954802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:41.960761 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:41.975125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 290/591] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:54:41 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/8adead3860f144099efa11cefa83a37f/roles/6328cebc97c34f12bc2c15bafa4ba1e0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:41.975125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.051134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62fefe-2c0a-4cad-b03b-8b19f26df432 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.101623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.102276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.102627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.102975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.103376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99050d4f-7c98-4f6e-b421-1a259dfc4a64 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.119718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.121997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.128161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:42.140414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:42.142342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:42.144143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:42.144143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:42.144143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:42.146945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 302/592] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:42.148443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.156446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.157079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.157360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.157637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.158924 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:42.159284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.161146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.178962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 291/593] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:42.180149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.258767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.276612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd94f3c-fdb3-4386-9804-471837197ec4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.304619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655dab05-b052-4c72-80df-9a0f998697bd None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.326353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.327002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.327330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.327607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.338480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.341395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.345317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f008c4-dbaf-43d3-8fea-8ff07147665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.345930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f008c4-dbaf-43d3-8fea-8ff07147665f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.346203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f008c4-dbaf-43d3-8fea-8ff07147665f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.346473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f008c4-dbaf-43d3-8fea-8ff07147665f None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.347138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f008c4-dbaf-43d3-8fea-8ff07147665f None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:42.347441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f008c4-dbaf-43d3-8fea-8ff07147665f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.348795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f008c4-dbaf-43d3-8fea-8ff07147665f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:42.355266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:42.357714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 303/594] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:42.360271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.375424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 292/595] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:42.375424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cef604-38fe-413e-b200-66747e404604 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.375424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cef604-38fe-413e-b200-66747e404604 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.375424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cef604-38fe-413e-b200-66747e404604 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.375424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 293/596] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:42.375424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cef604-38fe-413e-b200-66747e404604 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.413863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.485266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd120d-69d5-4d68-8323-9078b863c471 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.489612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cef604-38fe-413e-b200-66747e404604 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.521006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.521639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.521919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.522188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.530255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.530683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b0fdb-040c-41b0-88d0-c07e33d397e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.531312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b0fdb-040c-41b0-88d0-c07e33d397e1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.531596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b0fdb-040c-41b0-88d0-c07e33d397e1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.531758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.531852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b0fdb-040c-41b0-88d0-c07e33d397e1 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.532786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b0fdb-040c-41b0-88d0-c07e33d397e1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:42.533100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b0fdb-040c-41b0-88d0-c07e33d397e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.534793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b0fdb-040c-41b0-88d0-c07e33d397e1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:42.544666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:42.545599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:42.546506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:42.550655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 304/597] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:42.551542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.562711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 294/598] 10.222.0.94 () {56 vars in 1139 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:42.562711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47355972-fe49-457d-9818-537f343a7c75 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.562711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47355972-fe49-457d-9818-537f343a7c75 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.562711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47355972-fe49-457d-9818-537f343a7c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.566068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.584117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-47355972-fe49-457d-9818-537f343a7c75 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:42.585346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-47355972-fe49-457d-9818-537f343a7c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:42.663774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 295/599] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:42.664053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163a25e-6cbb-4b69-9308-2d7abed875e5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.665086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47355972-fe49-457d-9818-537f343a7c75 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.693607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.720259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.721622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.722329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.722777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.732801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.735025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:42.751815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:42.752959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:42.754021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:42.762008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 305/600] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:42.762008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.789630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47355972-fe49-457d-9818-537f343a7c75 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.844307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.844960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.845242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.845520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.846774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:42.847107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.849098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.863213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.865696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 296/601] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:42.866541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:42.906360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82fc3a-a135-46f6-b493-7c8194a28aa2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.942371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723341e-767d-451f-8262-591f080cba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:42.943443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723341e-767d-451f-8262-591f080cba99 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:42.943961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723341e-767d-451f-8262-591f080cba99 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:42.944475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723341e-767d-451f-8262-591f080cba99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:42.956970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723341e-767d-451f-8262-591f080cba99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:42.957574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:42.959201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723341e-767d-451f-8262-591f080cba99 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-4723341e-767d-451f-8262-591f080cba99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:42.975840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:42.976825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:42.977716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:42.977716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:42.977716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:42.978312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731c9e1-e158-40cd-aa02-d917ea4be7b3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:42.980605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 306/602] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:42.981915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723341e-767d-451f-8262-591f080cba99 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.018817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:43.019467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:43.019744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:43.020012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:43.020710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:43.021013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:43.022428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:43.041016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 297/603] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:43.041016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.094040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723341e-767d-451f-8262-591f080cba99 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.117000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:43.132270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:43.132933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:43.133211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:43.133484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/1d9d979b6539460e82759320c3d96a79/roles/6328cebc97c34f12bc2c15bafa4ba1e0` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:43.152542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=1d9d979b6539460e82759320c3d96a79, role_id=6328cebc97c34f12bc2c15bafa4ba1e0)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:43.154273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:43.155038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36883c43-937d-4b23-a49f-974cdc2564a7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.178438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 307/604] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:54:42 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/1d9d979b6539460e82759320c3d96a79/roles/6328cebc97c34f12bc2c15bafa4ba1e0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:43.178438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.198762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:43.199442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:43.199739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:43.200000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:43.216406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:43.221651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:43.222663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:43.223545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 298/605] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:43.224418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.228270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:43.341386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29aefa-5a01-4e88-8472-d778da1be7d2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.354716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c216d7-596d-4a1d-8aa2-4131525f1f91 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.400563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:43.401411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:43.401675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:43.402239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:43.402894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:43.403002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:43.403260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:43.403678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:43.403962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:43.404236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/ee56cf453075431ab2791e15e803d51c/roles/6328cebc97c34f12bc2c15bafa4ba1e0` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:43.404628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:43.420327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=ee56cf453075431ab2791e15e803d51c, role_id=6328cebc97c34f12bc2c15bafa4ba1e0)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:43.420542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 308/606] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:43.421360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.421978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:43.439727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 299/607] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:54:43 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/ee56cf453075431ab2791e15e803d51c/roles/6328cebc97c34f12bc2c15bafa4ba1e0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:43.440740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.497219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:43.575699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:43.647157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dbce6-1010-4afc-afbd-48d31e8de828 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.659658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae257b-d6f1-4058-9437-0781934cdad0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.686413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:43.687026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:43.687342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:43.687614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:43.688279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:43.688581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:43.689978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:43.699639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 309/608] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:43.700934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:43.705088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.710779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:43.711060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:43.711417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:43.720902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:43.722433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:43.737378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:43.738706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:43.739695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 300/609] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:43.747937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:43.902814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:43.921980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a9be-d5d7-4c43-8e3a-a271064f5079 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.942972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b84974-d383-4935-86bf-25546204bfb9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:43.971550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:43.999737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:44.000277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.000559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.000832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.001534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:44.001841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:44.003294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:44.005958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:44.006593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.006871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.007174 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.026546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:44.028695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:44.036581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 301/610] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:44.041545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:44.065240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:44.066270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:44.067219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:44.067219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:44.067219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:44.071268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 310/611] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:44.072172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:44.204353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3da264-cf5c-4102-a3b0-5454207b8ff0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:44.233298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef3bb9-6ab5-4334-bb3a-58e7c6923ec3 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:44.252604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b34e7-8c07-4f88-ad1b-00653f27bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:44.253397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b34e7-8c07-4f88-ad1b-00653f27bd2b None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.253683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b34e7-8c07-4f88-ad1b-00653f27bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.253960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b34e7-8c07-4f88-ad1b-00653f27bd2b None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/70c6682b92694484b2fa2710314f547f/roles/6328cebc97c34f12bc2c15bafa4ba1e0` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.281122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b34e7-8c07-4f88-ad1b-00653f27bd2b None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=70c6682b92694484b2fa2710314f547f, role_id=6328cebc97c34f12bc2c15bafa4ba1e0)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:44.282705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b34e7-8c07-4f88-ad1b-00653f27bd2b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:44.286858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:44.287891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.288212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.288593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.289225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:44.289589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:44.291027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:44.301420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 302/612] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:54:44 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/70c6682b92694484b2fa2710314f547f/roles/6328cebc97c34f12bc2c15bafa4ba1e0 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:44.307117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc70c51-5e53-48a3-9d97-e5236c3504bc None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.307117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc70c51-5e53-48a3-9d97-e5236c3504bc None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.307561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc70c51-5e53-48a3-9d97-e5236c3504bc None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.309819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 303/613] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:44.316110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af255f55-7b41-469c-85b9-1544db022779 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.316399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af255f55-7b41-469c-85b9-1544db022779 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.316660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af255f55-7b41-469c-85b9-1544db022779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.318424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 311/614] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:44.329002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:44.338022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:44.372438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-af255f55-7b41-469c-85b9-1544db022779 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:44.376985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-af255f55-7b41-469c-85b9-1544db022779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:44.406517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:44.477036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 304/615] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:44.532937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b0fe2-85c5-4347-8d0a-4c73d12e462d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:44.571041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:44.571705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.571983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.572256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.581111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:44.582604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:44.595963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:44.596980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:44.597954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 312/616] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:44.604933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af255f55-7b41-469c-85b9-1544db022779 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:44.746777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af255f55-7b41-469c-85b9-1544db022779 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:44.805037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:44.805981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.806452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.807245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.808778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:44.808910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:44.809399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:44.811252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:44.851133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 305/617] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Aug 29 18:54:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:44.857186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:44.879547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:44.892961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:44.939355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda7080-f2ee-4092-808b-7b4bbfa22fb1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:44.978392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:44.979210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:44.979503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:44.979802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:44.991276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:44.997449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:44.998638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:44.999716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:45.003462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 313/618] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:45.004764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.084178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b24340-3b3e-4616-818a-9806be649e47 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.116559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ab334-47fb-4267-b93c-55416d9986d9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.121052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.121701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.121988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.123456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.133134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.134600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:45.143456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.156364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.157011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.157305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.157586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.158260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:45.158566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.160004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:45.160654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:45.161803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:45.162867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:45.162867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:45.162867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:45.169438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 306/619] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:45.174958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 314/620] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:45.175287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.176023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.284044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f8676-af6b-4267-b9f0-5503e6d5651f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.320259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.320989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.321347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.321695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.334080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.335756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:45.351526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:45.352692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:45.353776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:45.353776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:45.353776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:45.354935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 315/621] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:45.360990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.368635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.411467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.474053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293152c-acd1-42c5-9f49-91a179479197 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.490602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782ce4b-807b-47e9-b6b0-a4eae7a7ba69 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.536029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.537028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.537543 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.538010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.538859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:45.539386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.541298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:45.555400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6fcb5-8d75-4e8e-be2f-0ad95e48f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.556032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6fcb5-8d75-4e8e-be2f-0ad95e48f327 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.556315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6fcb5-8d75-4e8e-be2f-0ad95e48f327 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.556590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6fcb5-8d75-4e8e-be2f-0ad95e48f327 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.557248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6fcb5-8d75-4e8e-be2f-0ad95e48f327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:45.558142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6fcb5-8d75-4e8e-be2f-0ad95e48f327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.558978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 316/622] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:45.564722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.566192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6fcb5-8d75-4e8e-be2f-0ad95e48f327 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:45.598367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 307/623] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:45.599898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3ef8-9c0f-44b8-a394-b8f8920ccf6b None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.600186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3ef8-9c0f-44b8-a394-b8f8920ccf6b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.600462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3ef8-9c0f-44b8-a394-b8f8920ccf6b None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.603187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 308/624] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:45.607434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3ef8-9c0f-44b8-a394-b8f8920ccf6b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.694504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff61137-e968-454e-9547-0b87122dd693 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.707611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.728984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3ef8-9c0f-44b8-a394-b8f8920ccf6b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:45.734362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d959cd-8c0a-4e60-b0cd-fd09a397180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.735000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d959cd-8c0a-4e60-b0cd-fd09a397180c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.735376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d959cd-8c0a-4e60-b0cd-fd09a397180c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.736021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d959cd-8c0a-4e60-b0cd-fd09a397180c None admin] PATH_INFO: `/v3/users/nova` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.745753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d959cd-8c0a-4e60-b0cd-fd09a397180c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.747463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d959cd-8c0a-4e60-b0cd-fd09a397180c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:45.756128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-01d959cd-8c0a-4e60-b0cd-fd09a397180c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:45.762596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:45.763932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:45.764928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:45.769406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 317/625] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:45.770817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd710e-e100-4e67-acfb-0ffd8eaff462 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.771197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd710e-e100-4e67-acfb-0ffd8eaff462 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.771476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd710e-e100-4e67-acfb-0ffd8eaff462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.774574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:45.775488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:45.775987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:45.776616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:45.784406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:45.786206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:45.791104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d5cd710e-e100-4e67-acfb-0ffd8eaff462 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:45.793243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d5cd710e-e100-4e67-acfb-0ffd8eaff462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:45.800336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:45.801435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:45.802346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:45.805297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 309/626] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:45.806331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.897061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 318/627] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:45.897061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd710e-e100-4e67-acfb-0ffd8eaff462 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:45.962201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc519ee-a1f1-41ed-97ac-ed87f2317d2a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.002652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd306adb-e74d-4d8f-bba9-f8255cb21308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.003298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd306adb-e74d-4d8f-bba9-f8255cb21308 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.003584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd306adb-e74d-4d8f-bba9-f8255cb21308 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.003864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd306adb-e74d-4d8f-bba9-f8255cb21308 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.004607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd306adb-e74d-4d8f-bba9-f8255cb21308 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:46.004919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd306adb-e74d-4d8f-bba9-f8255cb21308 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:46.006427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd306adb-e74d-4d8f-bba9-f8255cb21308 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:46.023253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:46.033053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 310/628] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:46.034949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db2f3c-d9a4-4a02-b472-eab3111d43a7 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.035868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db2f3c-d9a4-4a02-b472-eab3111d43a7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.036387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db2f3c-d9a4-4a02-b472-eab3111d43a7 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.038435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 311/629] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:46.043949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db2f3c-d9a4-4a02-b472-eab3111d43a7 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:46.048259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd710e-e100-4e67-acfb-0ffd8eaff462 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.091063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68aadd3-affb-49cd-8086-338e93e73f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.091746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aadd3-affb-49cd-8086-338e93e73f1f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.092030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aadd3-affb-49cd-8086-338e93e73f1f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.092306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aadd3-affb-49cd-8086-338e93e73f1f None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.093003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68aadd3-affb-49cd-8086-338e93e73f1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:46.093313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68aadd3-affb-49cd-8086-338e93e73f1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:46.094759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68aadd3-affb-49cd-8086-338e93e73f1f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:46.106322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:46.108147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 319/630] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:46.109651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2ff7-c5c7-4928-beb7-fdcc56d640a0 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.109949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2ff7-c5c7-4928-beb7-fdcc56d640a0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.110229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2ff7-c5c7-4928-beb7-fdcc56d640a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.130055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-069e2ff7-c5c7-4928-beb7-fdcc56d640a0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:46.131297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-069e2ff7-c5c7-4928-beb7-fdcc56d640a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:46.188975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db2f3c-d9a4-4a02-b472-eab3111d43a7 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.219319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 320/631] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:46.220790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2ff7-c5c7-4928-beb7-fdcc56d640a0 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:46.241585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.242217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.242500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.242775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.334983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:46.343715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:46.345189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:46.368447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2ff7-c5c7-4928-beb7-fdcc56d640a0 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.397327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:46.423861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b213392-e248-490e-a869-15978713622f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.425171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b213392-e248-490e-a869-15978713622f None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.425900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b213392-e248-490e-a869-15978713622f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.426788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b213392-e248-490e-a869-15978713622f None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/7a0e57abdb8d4966a2066a85bbfbc15e/roles/6328cebc97c34f12bc2c15bafa4ba1e0` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.447225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b213392-e248-490e-a869-15978713622f None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=7a0e57abdb8d4966a2066a85bbfbc15e, role_id=6328cebc97c34f12bc2c15bafa4ba1e0)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:46.450069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b213392-e248-490e-a869-15978713622f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:46.473653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 321/632] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:54:46 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/7a0e57abdb8d4966a2066a85bbfbc15e/roles/6328cebc97c34f12bc2c15bafa4ba1e0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:54:46.473653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b213392-e248-490e-a869-15978713622f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:46.492568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 312/633] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:46.496902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:46.626602 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b213392-e248-490e-a869-15978713622f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.634358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd4f21-66cb-4f43-b595-a5792547b3a5 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.662682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:46.675757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.676782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e0dec-7dbb-4ba1-af5d-929c2c5583f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.676921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.677451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.677554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e0dec-7dbb-4ba1-af5d-929c2c5583f6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.677754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e0dec-7dbb-4ba1-af5d-929c2c5583f6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.677942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.678049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e0dec-7dbb-4ba1-af5d-929c2c5583f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.686511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:46.689997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e0dec-7dbb-4ba1-af5d-929c2c5583f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:46.695809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6c7e0dec-7dbb-4ba1-af5d-929c2c5583f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:46.695954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:46.696870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 313/634] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:46.703534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaeb39-1405-4e3c-b6ce-d93a593af4ec None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.704316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaeb39-1405-4e3c-b6ce-d93a593af4ec None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.704316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaeb39-1405-4e3c-b6ce-d93a593af4ec None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.704316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 314/635] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:46.704972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaeb39-1405-4e3c-b6ce-d93a593af4ec None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:46.707116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:46.708223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:46.709249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:46.710048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 322/636] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:46.710888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:46.836826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaeb39-1405-4e3c-b6ce-d93a593af4ec None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.841447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8638a6-12f9-4451-94a3-48156a1039f2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:46.874498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.875193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.875782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.875963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.883269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783aa9d-e2dc-4a69-81af-e5f8809042db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:46.883891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783aa9d-e2dc-4a69-81af-e5f8809042db None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.884149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783aa9d-e2dc-4a69-81af-e5f8809042db None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.884417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783aa9d-e2dc-4a69-81af-e5f8809042db None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.885073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783aa9d-e2dc-4a69-81af-e5f8809042db None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:46.885406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783aa9d-e2dc-4a69-81af-e5f8809042db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:46.887939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783aa9d-e2dc-4a69-81af-e5f8809042db None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:46.888279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:46.889931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:46.896327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 323/637] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:46.898387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c94ce-7c4e-4ec8-9b50-dc0ec697afc6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:46.899461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c94ce-7c4e-4ec8-9b50-dc0ec697afc6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:46.899962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c94ce-7c4e-4ec8-9b50-dc0ec697afc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:46.906257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:46.907542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:46.909899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:46.909899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:46.909899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:46.909899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 315/638] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:46.909899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:46.924479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a43c94ce-7c4e-4ec8-9b50-dc0ec697afc6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:46.925670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a43c94ce-7c4e-4ec8-9b50-dc0ec697afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:46.970428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:46.979995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:47.012888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 324/639] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:47.014256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c94ce-7c4e-4ec8-9b50-dc0ec697afc6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.042847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d2a4-7ca7-4418-a979-6cdf09a7fee1 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.093436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.094450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.094957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.095465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.096333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:47.096853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.099087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.111794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 316/640] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:47.112928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.130371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c94ce-7c4e-4ec8-9b50-dc0ec697afc6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.174270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.175241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.175241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.175538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.188138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.189635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:47.205998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:47.206995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:47.208197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:47.208197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:47.208197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:47.211934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 325/641] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:47.212807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.246491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68f97f-3b80-4502-90a1-7d3177f51153 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.247836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:47.263042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:47.287642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.288636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.289118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.289589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.290476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:47.290785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.292751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.305630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 317/642] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:47.306298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.327742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ad5c6-631c-4c85-a370-11cf9372a47b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.368128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.368783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.369048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.369334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.380781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.382231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:47.397626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:47.398676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:47.399616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:47.399616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:47.399616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:47.399616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 326/643] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:47.399616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.422536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95b0b9-a7bf-49d1-a35b-fbf3fc1f7516 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.465713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.466333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.466631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.467114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.467755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:47.468064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.469430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.482533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 318/644] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:47.483298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.499923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:47.513435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0222d04-029b-4bee-be5f-cc887b317877 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.522954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:47.552602 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.553330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.553615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.553891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.564442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:47.569697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:47.570652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 327/645] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:47.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.606467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11edf8-1dac-4ea0-a73e-237a0c59efeb None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.646644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.647311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.647591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.647873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.655559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.657902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:47.675310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:47.676482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:47.677546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:47.681033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 319/646] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:47.681872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.693484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95cc3-d04d-40ee-903a-71ab09516faa None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.732929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.733684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.733823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.734069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.734704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:47.735004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.736462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.749215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 328/647] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:47.755452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.762260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:47.794620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:47.810782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa5b03-529e-4099-8c1f-d6d6e40771db None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.851833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.852508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.852739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.853015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.862952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.864685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:47.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:47.877662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:54:47.878624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:47.883017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276937b4-1cf8-4f3d-b09c-8979f7b40bc4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:47.885148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 320/648] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:47.886157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:47.922317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:47.922776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:47.923058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:47.923377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:47.924033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:47.924339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:47.925724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:47.935091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 329/649] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:47.935947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.014078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98365ed-3120-4fb9-9dbf-8d2af8e97cf4 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.014895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.054651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.055310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.055493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7142648-4498-4440-bd45-e0186e96e071 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.055598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.055859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.056569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:48.056880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.058580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.071368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 321/650] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:48.072213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.090402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.099051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.099704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.099981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.100251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] PATH_INFO: `/v3/users/glance` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.110494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.111899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:48.124438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:48.125464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 330/651] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:48.126409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.192268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b27eb-4307-40e5-8fba-6213500cd507 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.229469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.230170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.230450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.230721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.231627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:48.232161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.233492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.234685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fdff9-7f2b-4caa-b35f-09309ba04c61 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.257767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 322/652] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:48.257767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.265447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.275569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.276734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.277294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.277569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.289249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.290688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:48.306097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:48.307149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:48.308232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:48.308232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:48.308232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:48.308232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 331/653] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:48.308232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.336931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.372955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd8fb-2c54-457d-bd1a-7d02a32b1270 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.408888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.409496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.409780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.410045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.410770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=8adead3860f144099efa11cefa83a37f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:48.411096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.412499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.419224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 323/654] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/role_assignments?user.id=8adead3860f144099efa11cefa83a37f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:48.420041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.441141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96076-d62a-4dbe-b82c-704ebb0fe4a9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.493075 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.493838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.494021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.494290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.495039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:48.495671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.497201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.518721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 332/655] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:48.519739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.552524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bba540-5471-4d40-be48-39284e1f0a33 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.561860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.594535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.595189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.595471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.595746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.599548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.648833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832ac75-e812-453f-94a3-3c3d897e42df None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.680510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.682556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.686585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba65c54-9039-40ef-ae0f-6bce2806b209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.687387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba65c54-9039-40ef-ae0f-6bce2806b209 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.687629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba65c54-9039-40ef-ae0f-6bce2806b209 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.687904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba65c54-9039-40ef-ae0f-6bce2806b209 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.688548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba65c54-9039-40ef-ae0f-6bce2806b209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:48.688851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba65c54-9039-40ef-ae0f-6bce2806b209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.690214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba65c54-9039-40ef-ae0f-6bce2806b209 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.707253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 333/656] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:48.707253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd2af3-845d-4ace-b50d-44464bb8a1ae None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.707253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd2af3-845d-4ace-b50d-44464bb8a1ae None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.707253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd2af3-845d-4ace-b50d-44464bb8a1ae None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.711202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 334/657] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:48.711202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd2af3-845d-4ace-b50d-44464bb8a1ae None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.796740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 324/658] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:48.802088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:48.820051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.827882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd2af3-845d-4ace-b50d-44464bb8a1ae None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.846716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:48.870832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.871563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.871845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.872117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.917006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99361a-4638-47ec-8688-3e073d8c7e8e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:48.955111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.955111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.956157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0e3eb-df95-44b9-9d46-e5da155493f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:48.956868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0e3eb-df95-44b9-9d46-e5da155493f4 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.957150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0e3eb-df95-44b9-9d46-e5da155493f4 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.957425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0e3eb-df95-44b9-9d46-e5da155493f4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:48.965370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0e3eb-df95-44b9-9d46-e5da155493f4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:48.966869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0e3eb-df95-44b9-9d46-e5da155493f4 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-39b0e3eb-df95-44b9-9d46-e5da155493f4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:48.978668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:48.979665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 18:54:48.980612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:48.984994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 325/659] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:48.986368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e080657-205e-4747-8ab3-a639e786fd8b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:48.986650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e080657-205e-4747-8ab3-a639e786fd8b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:48.986925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e080657-205e-4747-8ab3-a639e786fd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.005654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5e080657-205e-4747-8ab3-a639e786fd8b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:49.006835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e080657-205e-4747-8ab3-a639e786fd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:49.060688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.067528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 335/660] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:49.068399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.095098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 326/661] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:49.095098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e080657-205e-4747-8ab3-a639e786fd8b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.139797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.182076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ce9bc-7567-4d37-8555-bf128f0278ba None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.226616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.227386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.227662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.227931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.241452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:49.241853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e080657-205e-4747-8ab3-a639e786fd8b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.242910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:49.265550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:49.266566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:49.267456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:49.267456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:49.267456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:49.267456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 336/662] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:49.268149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.291043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.292637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.293692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.294172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.295453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:49.296221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:49.298714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:49.324286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.326752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 327/663] 10.222.0.94 () {56 vars in 1139 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:49.328311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.390098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660b0f0-f89b-480b-814f-6cd37b1433e3 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.427417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.428136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.428446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.428729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.436830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.441979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:49.443650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:49.457431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3b7d3-1b3a-4585-94aa-a0683829029c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:49.459919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:49.460844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:49.461720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:49.461720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:49.461720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:49.461720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 337/664] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:49.461948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.499017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.499720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.500007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.500555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.501465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:49.501636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:49.502972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:49.521578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 328/665] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:49.521578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.579120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.583694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a724d-b45d-4910-b224-091485094a4f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.624996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.625630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.625916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.626193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.637441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c31c-85ca-4874-9cbd-0dcbb466ca3c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.674973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.675645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.675925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.676198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.687235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:49.692457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:49.693385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:49.694259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 329/666] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:49.694949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.699899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.711629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:49.713081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:49.810720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5673-af89-44b7-9c2a-14f954c54f0b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.825027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.834937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 338/667] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:49.835881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.850923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.851586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.851865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.852383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:49.852776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:49.853078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:49.855015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:49.868197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 330/668] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:49.868912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:49.944828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:49.960617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec18d3-9909-403f-b00c-ff2e065d8a46 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.987839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a742ba-2368-4247-bcb7-1f8577220237 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:49.998649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:49.999510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:49.999710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:49.999979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.007175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.008624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:50.020276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:50.021215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:50.022101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 339/669] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:50.022600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.030573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.031273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.031542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.031809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.032487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:50.032794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.034161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.047963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 331/670] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:50.047963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.098700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:50.148099 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e15f-0d09-4ead-9ed4-37a5b2530ebe None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.167813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c38ff5-17c7-4794-a366-62f067d636bb None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.185192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.185833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.186113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.186384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.197743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:50.198413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.199866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.208598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.209219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.209499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.209770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:50.216776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:50.218098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:50.219004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:50.219004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:50.219004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:50.220962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.222520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.222623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 340/671] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:50.223455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:50.234801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:50.235778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:50.236678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 332/672] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:50.238125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.341799 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06d6b-bd14-47ad-ac26-7242e491ec7e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.357255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd77b07-e166-4f48-8743-1da6bb5fb008 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.361288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:50.383554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.384162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.384440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.384714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.385940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:50.386250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.388180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.395708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.396085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.396360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.396629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.400942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 341/673] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:50.407250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.409399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.411302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:50.426319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:50.427304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:50.428201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:50.428201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:50.428201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:50.428201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 333/674] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:50.428201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.475447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:50.521861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd00abd-88c5-46b0-9b71-c0d6d1b95a5f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.546755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164423f-ac25-4690-a516-18cd90437e6f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.556350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac317194-0332-489b-ab19-992d46755968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.556963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac317194-0332-489b-ab19-992d46755968 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.557270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac317194-0332-489b-ab19-992d46755968 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.558084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac317194-0332-489b-ab19-992d46755968 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.558726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac317194-0332-489b-ab19-992d46755968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:50.559029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac317194-0332-489b-ab19-992d46755968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.560431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac317194-0332-489b-ab19-992d46755968 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.571860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 342/675] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:50.572700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac317194-0332-489b-ab19-992d46755968 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.587980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.589126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.589504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.589784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.590493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:50.590843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.592466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.605175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 334/676] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:50.606330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.611021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:50.678195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac317194-0332-489b-ab19-992d46755968 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.705798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:50.709526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904374c-19d2-4451-afc2-362ef11bf86d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.712597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d295e11-9e52-477e-9715-0112a77d59e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.713310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d295e11-9e52-477e-9715-0112a77d59e5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.714159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d295e11-9e52-477e-9715-0112a77d59e5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.714495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d295e11-9e52-477e-9715-0112a77d59e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.715211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d295e11-9e52-477e-9715-0112a77d59e5 None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=ee56cf453075431ab2791e15e803d51c) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:50.715526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d295e11-9e52-477e-9715-0112a77d59e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.716986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d295e11-9e52-477e-9715-0112a77d59e5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.723012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 343/677] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/role_assignments?user.id=ee56cf453075431ab2791e15e803d51c&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:50.724217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c74d38-610e-4feb-8428-bb6e2ba6fdfd None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.724501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c74d38-610e-4feb-8428-bb6e2ba6fdfd None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.724778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c74d38-610e-4feb-8428-bb6e2ba6fdfd None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.731332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 344/678] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:50.732371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c74d38-610e-4feb-8428-bb6e2ba6fdfd None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.752112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.752845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.753192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.753534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.754343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:50.754712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.756286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.774845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 335/679] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:50.781068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.856035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:50.867129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c74d38-610e-4feb-8428-bb6e2ba6fdfd None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.911619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106706c-de5e-4b5b-919d-05b933701e6a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:50.917812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e2bca-8633-4c53-8000-2c74a803e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.919128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e2bca-8633-4c53-8000-2c74a803e533 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.919128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e2bca-8633-4c53-8000-2c74a803e533 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.919128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e2bca-8633-4c53-8000-2c74a803e533 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.927615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e2bca-8633-4c53-8000-2c74a803e533 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.929133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e2bca-8633-4c53-8000-2c74a803e533 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-cd2e2bca-8633-4c53-8000-2c74a803e533 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:50.944568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:50.945691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:50.946975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 345/680] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:50.948427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dbd70-ae70-4a4f-8076-f6c103bd9d45 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.948904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dbd70-ae70-4a4f-8076-f6c103bd9d45 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.949353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dbd70-ae70-4a4f-8076-f6c103bd9d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.953419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:50.954169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:50.954549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:50.954915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:50.955786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=1d9d979b6539460e82759320c3d96a79) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:50.956170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:50.957592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:50.964030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 336/681] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/role_assignments?user.id=1d9d979b6539460e82759320c3d96a79&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:50.966802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:50.975442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a45dbd70-ae70-4a4f-8076-f6c103bd9d45 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:50.976736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a45dbd70-ae70-4a4f-8076-f6c103bd9d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:50.997235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:51.078055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 346/682] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:51.078996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dbd70-ae70-4a4f-8076-f6c103bd9d45 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:51.093722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681f15-6089-44b6-99fe-82b54622de20 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:51.179289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:51.180541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:51.181052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:51.181419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:51.194693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:51.196415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:51.230148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:51.231363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:51.232477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:51.234364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 337/683] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:51.242683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:51.244519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:51.276955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dbd70-ae70-4a4f-8076-f6c103bd9d45 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:51.352498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:51.353888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:51.354715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:51.356030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:51.357527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:51.358182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:51.361726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:51.384681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 347/684] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:51.392508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:51.402527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:51.434066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b69bed-2188-4aac-95f9-5958b1c01a88 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:51.473030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:51.473722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:51.474013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:51.474354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:51.475135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:51.475447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:51.476885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:51.498483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 338/685] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Aug 29 18:54:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:51.502133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:51.611564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41774f0e-ba15-4ec1-be57-119638a4cf23 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:51.628705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d4d06-3f9a-4e1e-88d4-690718d054cd None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:51.654339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:51.655133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:51.655470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:51.655963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:51.669743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:51.675106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:51.682660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:51.685044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 339/686] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:51.687628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:51.716574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:51.718231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:51.719240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:51.720155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:51.721989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=70c6682b92694484b2fa2710314f547f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:51.722941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:51.726379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:51.740159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 348/687] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:51 2022] GET /identity/v3/role_assignments?user.id=70c6682b92694484b2fa2710314f547f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 608 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:51.742250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:51.771444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:51.874945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8eeb30-9cef-42fd-ad56-618b0691c1ac None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:51.953928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:51.954909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:51.955472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:51.955968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:51.986986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfcba5-807c-42eb-af12-b0a9711e68dd None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:52.043757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:52.045013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:52.045529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:52.046011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:52.046947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:52.047546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.049736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.062160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 349/688] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:52.064536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:52.068140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.069938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.073416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:52.196039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:52.241293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e090d0-ee54-41dd-a833-a20313974664 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:52.254289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 340/689] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:52.259864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:52.309981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:52.311016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:52.311547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:52.312122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:52.337738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.340109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.371625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16da6e-5085-4533-a771-3f081e9080ba None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:52.372562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:52.373475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:52.374363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:52.374363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:52.374363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:52.383285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:52.383462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 350/690] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:52.384968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:52.423836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:52.424472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:52.424748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:52.425014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:52.443227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.444873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:52.469013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:52.470192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:52.471355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:52.471355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:52.471355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:52.471355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 341/691] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:52.481091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:52.561330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a232-ac2c-4fa8-94f7-71683aa8e7f9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:52.594782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:52.610662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:52.611320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:52.612024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:52.612307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:52.613264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:52.613574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.615762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.629540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 351/692] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:52.635990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:52.667826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222e11f-b7af-4228-9c6c-dd08de005f86 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:52.745234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:52.745857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:52.746139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:52.746414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:52.747084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:52.747432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.748875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.767186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 342/693] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:52.768636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:52.784285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:52.809469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bb6d1-54e3-43f3-9421-dc2eb8e3987b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:52.878381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d06e8-2e1d-494c-bb24-b374c02cf254 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:52.907837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:52.908498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:52.908786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:52.909066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:52.917583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:52.918142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:52.918425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:52.918716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:52.925945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.927445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.928195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:52.929721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:52.940233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:52.941174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:52.942073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 343/694] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:52.947208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:52.960366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:52.961371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:52.962300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 352/695] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:52.969384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:53.035725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:53.061416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916646b4-b9b7-4f43-a085-8a340e8bb813 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:53.068155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:53.100342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:53.100965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.101319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.101672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.102452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:53.102830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:53.104493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:53.118661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 344/696] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:53.125897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:53.180245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d24171-1267-4eb5-85ed-9da69c24cea1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:53.224253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85143102-f9c0-4b2e-b97c-22ce6eff4911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:53.225271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143102-f9c0-4b2e-b97c-22ce6eff4911 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.225767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143102-f9c0-4b2e-b97c-22ce6eff4911 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.226566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85143102-f9c0-4b2e-b97c-22ce6eff4911 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.227867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85143102-f9c0-4b2e-b97c-22ce6eff4911 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:53.228667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85143102-f9c0-4b2e-b97c-22ce6eff4911 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:53.231291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85143102-f9c0-4b2e-b97c-22ce6eff4911 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:53.254046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 353/697] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:54:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:53.254354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fdfe0-8e91-42d9-a9b0-00861d892d41 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:53.255858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287783d-c858-4c6d-8640-ed0165b9695f None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.256496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287783d-c858-4c6d-8640-ed0165b9695f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.256953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287783d-c858-4c6d-8640-ed0165b9695f None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.259718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 354/698] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:53.261201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287783d-c858-4c6d-8640-ed0165b9695f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:53.299989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:53.307806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471da8bc-228a-467c-a29f-4451fc145d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:53.308437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471da8bc-228a-467c-a29f-4451fc145d3a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.309609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471da8bc-228a-467c-a29f-4451fc145d3a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.309976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471da8bc-228a-467c-a29f-4451fc145d3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.310732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471da8bc-228a-467c-a29f-4451fc145d3a None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=7a0e57abdb8d4966a2066a85bbfbc15e) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:53.311122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471da8bc-228a-467c-a29f-4451fc145d3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:53.312905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471da8bc-228a-467c-a29f-4451fc145d3a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:53.327113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 345/699] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:53 2022] GET /identity/v3/role_assignments?user.id=7a0e57abdb8d4966a2066a85bbfbc15e&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:53.327113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2f8c-3414-4289-bb80-7f24f229826c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.327113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2f8c-3414-4289-bb80-7f24f229826c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.327113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2f8c-3414-4289-bb80-7f24f229826c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.347956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b9fd2f8c-3414-4289-bb80-7f24f229826c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:53.349922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b9fd2f8c-3414-4289-bb80-7f24f229826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:53.465699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287783d-c858-4c6d-8640-ed0165b9695f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:53.471451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:53.480284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 346/700] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:53.481768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1bb66-0190-4c47-9d99-9cf3879342e4 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.482122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1bb66-0190-4c47-9d99-9cf3879342e4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.482809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1bb66-0190-4c47-9d99-9cf3879342e4 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.488796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 347/701] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:53.490667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c9ca5-04a3-41b5-ad15-8d3077794f23 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.490960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c9ca5-04a3-41b5-ad15-8d3077794f23 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.491272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c9ca5-04a3-41b5-ad15-8d3077794f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.517264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8b4c9ca5-04a3-41b5-ad15-8d3077794f23 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:53.518460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8b4c9ca5-04a3-41b5-ad15-8d3077794f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:53.526303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:53.527316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.527918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.528397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.607632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 348/702] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:53.614858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:53.616649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:53.728049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 355/703] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:53.733253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c9ca5-04a3-41b5-ad15-8d3077794f23 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:53.796397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:53.821476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:53.824704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:53.850738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c9ca5-04a3-41b5-ad15-8d3077794f23 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:53.892107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:53.892762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:53.893040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:53.893314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:53.908394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:53.910265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:53.929163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:53.930236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:53.931605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:53.931605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:53.931605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:53.934314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 349/704] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:53.939558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:53.952132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca290b1d-69e9-42bd-b43c-c489babd2980 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.006011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.006569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.006849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.007158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.007749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.009115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:54.024917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:54.026037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:54.027331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 356/705] 10.222.0.94 () {56 vars in 1129 bytes} [Mon Aug 29 18:54:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:54.028139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.076955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5575515-6a5a-4eba-a674-17ff249e42eb None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.086489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:54.120770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.121390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.121673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.121952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.122612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:54.122920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.124339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.138878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 350/706] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:54.139845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.140708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:54.209623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20422f-e6ad-4f8c-b35b-6923272f5340 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.248760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.249401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.249683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.249960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.262286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.263702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:54.278667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:54.280834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:54.282886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:54.282886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:54.282886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:54.284668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 357/707] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:54.285506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.302624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08face-2a87-4c82-8d38-214a93f572ea None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.357880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ede6-4dfc-41f5-b5f9-ad66b04d6879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.358596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ede6-4dfc-41f5-b5f9-ad66b04d6879 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.358882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ede6-4dfc-41f5-b5f9-ad66b04d6879 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.359195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ede6-4dfc-41f5-b5f9-ad66b04d6879 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.359920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ede6-4dfc-41f5-b5f9-ad66b04d6879 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:54.360232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ede6-4dfc-41f5-b5f9-ad66b04d6879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.361608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ede6-4dfc-41f5-b5f9-ad66b04d6879 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.366685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 351/708] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Aug 29 18:54:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:54.367894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e650d-4777-42da-a493-f114874023ec None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.368179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e650d-4777-42da-a493-f114874023ec None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.368455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e650d-4777-42da-a493-f114874023ec None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.371184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 352/709] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:54.371184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e650d-4777-42da-a493-f114874023ec None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.407672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8d66f-e3c9-4866-b595-e21f54a95a5c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.412628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:54.431544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:54.452111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.452857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.453146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.453421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.461214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.462685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:54.476127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:54.477246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:54.478142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 358/710] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:54.478618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.501161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e650d-4777-42da-a493-f114874023ec None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.541469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676f1f2-dad9-4709-8301-684bf1baef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.542533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f1f2-dad9-4709-8301-684bf1baef64 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.542816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f1f2-dad9-4709-8301-684bf1baef64 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.543110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f1f2-dad9-4709-8301-684bf1baef64 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.543765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676f1f2-dad9-4709-8301-684bf1baef64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:54.544070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676f1f2-dad9-4709-8301-684bf1baef64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.545449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676f1f2-dad9-4709-8301-684bf1baef64 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.557633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 353/711] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:54.558942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30635-53f3-42cf-992d-4df3c8c30399 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.559264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30635-53f3-42cf-992d-4df3c8c30399 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.559652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30635-53f3-42cf-992d-4df3c8c30399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.586482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5df30635-53f3-42cf-992d-4df3c8c30399 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:54.588139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5df30635-53f3-42cf-992d-4df3c8c30399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:54.621731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa765-32c9-4c8e-8a68-a1047581d8c2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.661168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.661785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.662068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.662713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.662983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:54.663323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.664773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.683113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 359/712] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Aug 29 18:54:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:54.683113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.690967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 354/713] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:54.691958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df30635-53f3-42cf-992d-4df3c8c30399 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.727443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:54.754556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:54.882681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe2da9-925a-4e9c-81a6-b12d6d63e0d2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.893479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df30635-53f3-42cf-992d-4df3c8c30399 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:54.927552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.928184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.928585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.928864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.929583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:54.929917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.942197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:54.955660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 360/714] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:54.955982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:54.957407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:54.957407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:54.962221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:54.963003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:54.964459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:54.965101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:55.003098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 355/715] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:54:54 2022] POST /identity/v3/users => generated 254 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:55.009261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:55.121544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04523674-e97d-4fe4-9611-9bb0e4297b0b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:55.123556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:55.165718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:55.175556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:55.175795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:55.176745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:55.177031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:55.177789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=8adead3860f144099efa11cefa83a37f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:55.178263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:55.180205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:55.189604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 361/716] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:54:54 2022] GET /identity/v3/role_assignments?user.id=8adead3860f144099efa11cefa83a37f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:55.204236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cc855-ffa6-4c56-9178-4020fecc622c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:55.249457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:55.250563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:55.251051 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:55.251575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] PATH_INFO: `/v3/services/network` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:55.252396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:55.254058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:55.267442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:55.268525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:54:55.269402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:55.272614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 356/717] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:55 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:54:55.280592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:55.415469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c43c87-76cd-4170-9733-82ff329a1723 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:55.457376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6363f0-829d-467e-8277-7c99ada93346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:55.458363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6363f0-829d-467e-8277-7c99ada93346 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:55.458870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6363f0-829d-467e-8277-7c99ada93346 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:55.459363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6363f0-829d-467e-8277-7c99ada93346 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:55.460319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6363f0-829d-467e-8277-7c99ada93346 None admin] RBAC: Adding query filter params (name=network) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:55.460820 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6363f0-829d-467e-8277-7c99ada93346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:55.462382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6363f0-829d-467e-8277-7c99ada93346 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:55.469360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 362/718] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:54:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:55.476394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:55.606141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dad199-f00c-4038-ac04-f58ac69d4293 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:55.661164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ffec2-759f-4f45-873d-441e32fdeacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:55.661786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffec2-759f-4f45-873d-441e32fdeacb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:55.662066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffec2-759f-4f45-873d-441e32fdeacb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:55.662340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffec2-759f-4f45-873d-441e32fdeacb None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:55.662983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ffec2-759f-4f45-873d-441e32fdeacb None admin] RBAC: Adding query filter params (type=network) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:55.663314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ffec2-759f-4f45-873d-441e32fdeacb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:55.664676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ffec2-759f-4f45-873d-441e32fdeacb None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:55.676357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 357/719] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:54:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:55.931368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc155f-bf16-4103-a6c1-37adbd9fb856 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:55.931885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc155f-bf16-4103-a6c1-37adbd9fb856 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:55.931885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc155f-bf16-4103-a6c1-37adbd9fb856 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:55.934309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 363/720] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:55.939709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15840a-0457-49f8-b182-543b3f8f8117 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:55.939998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15840a-0457-49f8-b182-543b3f8f8117 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:55.940276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15840a-0457-49f8-b182-543b3f8f8117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:55.956086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:55.961849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-dd15840a-0457-49f8-b182-543b3f8f8117 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:55.963148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-dd15840a-0457-49f8-b182-543b3f8f8117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:56.039460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 358/721] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:56.480756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc155f-bf16-4103-a6c1-37adbd9fb856 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:56.517324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:56.613089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc155f-bf16-4103-a6c1-37adbd9fb856 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:56.662469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:56.663119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:56.663402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:56.663674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:56.680401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:56.686104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:56.687658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:56.689028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 364/722] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:56.693661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15840a-0457-49f8-b182-543b3f8f8117 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:56.869189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15840a-0457-49f8-b182-543b3f8f8117 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:56.921471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:56.922097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:56.922380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:56.922654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:56.923354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:56.923661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:56.925031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:56.932812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:56.943420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 359/723] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:54:56.945484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:57.052974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:57.068232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479da77-6489-4d07-88a2-2e61582774b7 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:57.129161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85935326-1719-4b0a-9451-616d22c7eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:57.130144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85935326-1719-4b0a-9451-616d22c7eec2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.130653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85935326-1719-4b0a-9451-616d22c7eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.131175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85935326-1719-4b0a-9451-616d22c7eec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.152042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85935326-1719-4b0a-9451-616d22c7eec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:57.153865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85935326-1719-4b0a-9451-616d22c7eec2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-85935326-1719-4b0a-9451-616d22c7eec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:57.176629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:57.177717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:57.178612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:57.178612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:57.178612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:57.178612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 365/724] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:57.183485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:57.210255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57f722-9854-413e-83ef-a777d12dba24 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.210929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57f722-9854-413e-83ef-a777d12dba24 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.211474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57f722-9854-413e-83ef-a777d12dba24 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.213278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 366/725] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:54:57.219810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37ed9f-d594-4c6d-b85f-38dd34e50f85 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.220458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37ed9f-d594-4c6d-b85f-38dd34e50f85 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.221060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37ed9f-d594-4c6d-b85f-38dd34e50f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.242556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7f37ed9f-d594-4c6d-b85f-38dd34e50f85 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:57.244279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7f37ed9f-d594-4c6d-b85f-38dd34e50f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:57.348482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542aab09-98bf-4fdd-8b8e-41fc6d121d85 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:57.391468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:57.392426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.392923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.393390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.394914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:57.395446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:57.397082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:57.411872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 360/726] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:57.420205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:57.425434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 367/727] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:57.517527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:57.571873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af60da-ed3d-4416-a846-3ab79806b16b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:57.594461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb0d6a-aee5-42ff-b623-54ee1bce1135 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.594834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb0d6a-aee5-42ff-b623-54ee1bce1135 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.595019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb0d6a-aee5-42ff-b623-54ee1bce1135 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.596685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 368/728] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:54:57.603054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dae4c-093b-4050-91a5-e6d0ecbf9eec None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.603354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dae4c-093b-4050-91a5-e6d0ecbf9eec None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.603642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dae4c-093b-4050-91a5-e6d0ecbf9eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.632063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eb42c-b33c-4eba-b683-179c90ef7f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:57.632272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a11dae4c-093b-4050-91a5-e6d0ecbf9eec None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:57.632763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb42c-b33c-4eba-b683-179c90ef7f7e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.633057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb42c-b33c-4eba-b683-179c90ef7f7e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.633310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a11dae4c-093b-4050-91a5-e6d0ecbf9eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:57.633408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb42c-b33c-4eba-b683-179c90ef7f7e None admin] PATH_INFO: `/v3/users/swift` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.642405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eb42c-b33c-4eba-b683-179c90ef7f7e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:57.643977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eb42c-b33c-4eba-b683-179c90ef7f7e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-313eb42c-b33c-4eba-b683-179c90ef7f7e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:57.658113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:57.659375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:57.660942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 361/729] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:57.661900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023cf7-3396-42cd-93a2-fbf13c6e955a None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.661900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023cf7-3396-42cd-93a2-fbf13c6e955a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.661900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023cf7-3396-42cd-93a2-fbf13c6e955a None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.663165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 362/730] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:57.668197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023cf7-3396-42cd-93a2-fbf13c6e955a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:57.733599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 369/731] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 18:54:57.735476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70647989-a1c1-4f85-987a-25613c7e3d5b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.735769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70647989-a1c1-4f85-987a-25613c7e3d5b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.736047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70647989-a1c1-4f85-987a-25613c7e3d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.743826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:57.761756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-70647989-a1c1-4f85-987a-25613c7e3d5b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:57.763348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-70647989-a1c1-4f85-987a-25613c7e3d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:57.803329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023cf7-3396-42cd-93a2-fbf13c6e955a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:57.828406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:57.861315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:57.862793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:57.863370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:57.863820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:57.864897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:57.865415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:57.866190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 370/732] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:57.867279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70647989-a1c1-4f85-987a-25613c7e3d5b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:57.867404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:57.897707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 363/733] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Aug 29 18:54:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:57.905335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:58.041983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70647989-a1c1-4f85-987a-25613c7e3d5b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:58.087167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:58.095764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:58.096740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.097233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.097702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.098604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:58.100305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:58.137264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 371/734] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:54:57 2022] POST /identity/v3/services => generated 251 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:58.140554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:58.155133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc02c18-6729-41b0-b9f6-33ba1fa90915 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:58.236066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:58.236720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.237004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.237280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.296034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8bec8-44d0-4602-bd9d-fe17274e5dcf None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:58.301766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:58.342789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:58.343400 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.343814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.343986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.350246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:58.352761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:58.358271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:58.363736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:58.365932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 372/735] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:54:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:58.368002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:58.445887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:58.525874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841357e1-20af-41b0-9744-79f14d246a46 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:58.546267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 364/736] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:54:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 18:54:58.547753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:58.568961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:58.568961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.568961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.571817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.584347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:54:58.590797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:58.591991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 373/737] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:58.593086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:58.652691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:58.695796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fde01f-9ce0-434f-9280-5844de66c911 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:58.733130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db146af-aee8-4531-a68e-14c0ec14a4ca None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:58.745804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419f8ff-e255-4928-9d3c-3a66ff6847c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:58.747003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419f8ff-e255-4928-9d3c-3a66ff6847c4 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.747591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419f8ff-e255-4928-9d3c-3a66ff6847c4 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.748126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419f8ff-e255-4928-9d3c-3a66ff6847c4 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.748992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419f8ff-e255-4928-9d3c-3a66ff6847c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:58.749521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:58.750020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419f8ff-e255-4928-9d3c-3a66ff6847c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:58.752593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419f8ff-e255-4928-9d3c-3a66ff6847c4 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:58.764569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 365/738] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:54:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:58.766124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d11ab-d30f-4039-a287-fbfb25fade0b None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.766557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d11ab-d30f-4039-a287-fbfb25fade0b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.766979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d11ab-d30f-4039-a287-fbfb25fade0b None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.768803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 366/739] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:54:58.769641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d11ab-d30f-4039-a287-fbfb25fade0b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:58.787217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cccffe-2332-4be6-8173-8cef30c1ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:58.787903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cccffe-2332-4be6-8173-8cef30c1ee50 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.788194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cccffe-2332-4be6-8173-8cef30c1ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.788467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cccffe-2332-4be6-8173-8cef30c1ee50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.800359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cccffe-2332-4be6-8173-8cef30c1ee50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:58.801861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cccffe-2332-4be6-8173-8cef30c1ee50 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-09cccffe-2332-4be6-8173-8cef30c1ee50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:58.820243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:58.821240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:58.822104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:58.822104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:58.822104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:58.822104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 374/740] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:58.822690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e60e65-2990-49ee-bd77-f489fe9aec2d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.822841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e60e65-2990-49ee-bd77-f489fe9aec2d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.823786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e60e65-2990-49ee-bd77-f489fe9aec2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.846558 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-14e60e65-2990-49ee-bd77-f489fe9aec2d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:54:58.847799 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-14e60e65-2990-49ee-bd77-f489fe9aec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:54:58.923405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d11ab-d30f-4039-a287-fbfb25fade0b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:58.950506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 375/741] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:54:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 18:54:58.951664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e60e65-2990-49ee-bd77-f489fe9aec2d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:58.977848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:58.978901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:58.980146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:58.981248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:58.982529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:58.983039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:58.985639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:58.988134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:58.998816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 367/742] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:59.000523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.088456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:59.097470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e60e65-2990-49ee-bd77-f489fe9aec2d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.137934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.138730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.139317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.139480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.150384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893ae4a-94ef-4947-bbd7-ae121603ea41 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.153219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.155106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:59.171614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:59.172836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:59.173961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:59.173961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:59.173961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:59.179115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 376/743] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:59.179115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.208604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.209292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.209581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.209853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.210572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:59.210884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.213235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.233065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 368/744] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:59.234382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.306556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a79441-82d3-4e76-91bb-af6e4292046c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.313795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:59.350905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.351570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.351849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.352124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.365036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.366519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.368466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438321-7651-48cb-8af7-c4fc7b1f9986 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.374868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:54:59.384248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:59.385542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:54:59.387007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:54:59.387007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:54:59.387007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:59.387007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 377/745] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:59.387007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.421214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.422372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.422833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.423345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.424237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:59.424676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.426559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.444383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 369/746] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:59.445229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.524947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2011007-e4b2-41ca-814e-3d291ab9792b None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.567878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.568402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c43b7c-9fd2-48b1-9431-23071c06915b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.568516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.568731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.569009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.578967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.581058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.587858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:54:59.595934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:59.596880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:54:59.597762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 378/747] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:59.598243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.617326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.618052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.618357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.618604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.619504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.621208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.651744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 370/748] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:54:59 2022] POST /identity/v3/services => generated 240 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:54:59.651744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.704653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:54:59.769131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fb077-a801-40b7-a1b0-8fcc706cf2a2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.805311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186ce41-3de2-4831-849c-1fca72ad668a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:54:59.815985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.817195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.817712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.818513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.819581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:54:59.819924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.821503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.837509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 379/749] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:54:59.839278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.861353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:54:59.862010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:54:59.862299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:54:59.862575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:54:59.875962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:54:59.878773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:54:59.900279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:54:59.901883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:54:59.902860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 371/750] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:54:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:54:59.904016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:54:59.953240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:00.011853 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6a708-ce00-4340-9cc9-1ca5d0866607 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:00.037884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:00.057168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf47cd-0f58-490a-9bae-5de847164340 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:00.060806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:00.061746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.062029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.062306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.063012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:00.063355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:00.064828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:00.081942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 380/751] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:54:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:00.082845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:00.111059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:00.113242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.113242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.113242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] PATH_INFO: `/v3/users/nova` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.124445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:00.127555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:00.145979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:00.146976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:00.147985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:00.149544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 372/752] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:54:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:00.152040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:00.264482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:00.274454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824bff-ab05-4244-b548-2fa90ba60404 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:00.316343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:00.316968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.317251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.317530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.318276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=7a0e57abdb8d4966a2066a85bbfbc15e) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:00.318586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:00.320065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:00.335394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 373/753] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:00 2022] GET /identity/v3/role_assignments?user.id=7a0e57abdb8d4966a2066a85bbfbc15e&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:00.336334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:00.354020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71414554-2422-4e4b-86a3-6c35e970cb11 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:00.397366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10795f40-c3cb-4639-872b-ac4a75cadcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:00.398265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10795f40-c3cb-4639-872b-ac4a75cadcba None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.398703 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10795f40-c3cb-4639-872b-ac4a75cadcba None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.399329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10795f40-c3cb-4639-872b-ac4a75cadcba None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.400283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10795f40-c3cb-4639-872b-ac4a75cadcba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:00.400750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10795f40-c3cb-4639-872b-ac4a75cadcba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:00.402782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10795f40-c3cb-4639-872b-ac4a75cadcba None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:00.423883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 381/754] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Aug 29 18:55:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:00.425540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f91a-4d4c-4863-9970-a7da1cfa51b7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.426070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f91a-4d4c-4863-9970-a7da1cfa51b7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.426550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f91a-4d4c-4863-9970-a7da1cfa51b7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.428474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 382/755] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:00.437876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ba53-9f17-42f2-b70d-36aaebdb6155 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.438141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ba53-9f17-42f2-b70d-36aaebdb6155 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.438768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ba53-9f17-42f2-b70d-36aaebdb6155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.466132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9aad7-466c-4f59-8943-5c98be704988 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:00.471840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1397ba53-9f17-42f2-b70d-36aaebdb6155 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:00.473770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1397ba53-9f17-42f2-b70d-36aaebdb6155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:00.513454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:00.514088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.514371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.514647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.515430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:00.515738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:00.517236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:00.539198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 374/756] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:55:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:00.540073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:00.542495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:00.576131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:00.603236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 383/757] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:00.604635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ba53-9f17-42f2-b70d-36aaebdb6155 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:00.731955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7679b-c411-4cfb-a722-3239d1ff8918 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:00.808622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:00.809723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.810207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.810783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.841390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ba53-9f17-42f2-b70d-36aaebdb6155 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:00.910755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:00.911690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:00.912031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:00.912758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:00.915013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:00.917486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:00.925418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:00.926900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:00.936477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:00.949289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:00.950305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:00.951303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:00.951303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:00.951303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:00.951303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 384/758] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:00.951552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:00.977898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:01.061722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 375/759] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:01.063532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.111671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039d9e3-ce17-4e9d-8d2d-92bc0b7a14a8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:01.169289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:01.170076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:01.170457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:01.170831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:01.172032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:01.173966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:01.188057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:01.189260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:55:01.190090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:01.192876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 385/760] 10.222.0.94 () {56 vars in 1129 bytes} [Mon Aug 29 18:55:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:01.194008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.217953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dce98-4cf7-4646-ab49-f55f72e6d1d0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:01.238776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:01.272277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:01.273620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:01.274713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:01.275256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:01.276734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:01.277246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:01.280390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:01.297425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 376/761] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:01.298923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.309574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:01.334059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d225702-5281-4d3c-adf1-3de1572c894a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:01.395041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:01.395740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:01.396010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:01.396298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:01.411276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:01.412186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:01.434095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:01.436095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:01.437587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:01.437587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:01.437587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:01.437587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 386/762] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:01.437587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.470597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4f0a9-81a4-4f0c-bbe1-7690b6ea90ac None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:01.545529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:01.546840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:01.547227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:01.547505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:01.548919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:01.549396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:01.551319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:01.563119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 377/763] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:01.563119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.618046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:01.645391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367cefc-031e-43fe-ba76-9df836d174be None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:01.697208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:01.698821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:01.699373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:01.699856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:01.703563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:01.714957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:01.716691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:01.729830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:01.731230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:01.732441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:01.737874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cf00f-ec57-4904-a55d-32ee5c02b0b7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:01.738576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 387/764] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:01.739627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.781756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:01.782647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:01.783201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:01.783647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:01.784541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:01.785077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:01.786813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:01.808899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 378/765] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:01.810189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.874656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e196a-add2-4143-b426-555f1ccd9692 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:01.939949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:01.940966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:01.941360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:01.941725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:01.942487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:01.942880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:01.944480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:01.951030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:01.957660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 388/766] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:01.957660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:01.962096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da42b04-c79a-484a-a5d8-55b6e9a0b3ae None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:02.019427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:02.020049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.020335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.020613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.021319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:02.021630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:02.024853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:02.038072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:02.039256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 379/767] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:02.040180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:02.110310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b65dc-d810-47db-abc0-acb00298400a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:02.151894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7bdc8-e910-4d39-a1f1-ca5c5f50a1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:02.152697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7bdc8-e910-4d39-a1f1-ca5c5f50a1e8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.152956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7bdc8-e910-4d39-a1f1-ca5c5f50a1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.153226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7bdc8-e910-4d39-a1f1-ca5c5f50a1e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.161071 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7bdc8-e910-4d39-a1f1-ca5c5f50a1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:02.162518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7bdc8-e910-4d39-a1f1-ca5c5f50a1e8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-bdb7bdc8-e910-4d39-a1f1-ca5c5f50a1e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:02.175501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:02.177360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 389/768] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85498-8103-48bd-b87d-bcf36a3f32a1 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.178521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85498-8103-48bd-b87d-bcf36a3f32a1 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.179137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85498-8103-48bd-b87d-bcf36a3f32a1 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.180245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 390/769] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:02.181038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e85498-8103-48bd-b87d-bcf36a3f32a1 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:02.195820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cd464-dcb3-4f90-a798-c0cbcc0970c8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:02.247644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:02.248639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.249090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.249399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.250436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] RBAC: Adding query filter params (interface=public, service_id=cb8fde3c5b884089aac07b9aaa3b6ed5, region_id=RegionOne) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:02.251138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:02.252624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:02.263012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 380/770] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:55:02 2022] GET /identity/v3/endpoints?service_id=cb8fde3c5b884089aac07b9aaa3b6ed5&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:02.263928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:02.292778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:02.320308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e85498-8103-48bd-b87d-bcf36a3f32a1 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:02.340976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:02.364586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaff7c-ae4b-409e-bcaa-e393df9a28f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:02.365671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaff7c-ae4b-409e-bcaa-e393df9a28f4 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.366132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaff7c-ae4b-409e-bcaa-e393df9a28f4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.366581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaff7c-ae4b-409e-bcaa-e393df9a28f4 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/8adead3860f144099efa11cefa83a37f/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.369385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaff7c-ae4b-409e-bcaa-e393df9a28f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=8adead3860f144099efa11cefa83a37f, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:02.370928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaff7c-ae4b-409e-bcaa-e393df9a28f4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:02.388767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 391/771] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:55:02 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/8adead3860f144099efa11cefa83a37f/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:55:02.389841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038a2de-ea17-4164-95a0-b3bbd97b40d2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.390114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038a2de-ea17-4164-95a0-b3bbd97b40d2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.390412 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038a2de-ea17-4164-95a0-b3bbd97b40d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.412936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e038a2de-ea17-4164-95a0-b3bbd97b40d2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:02.414326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e038a2de-ea17-4164-95a0-b3bbd97b40d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:02.421316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a1406-179a-47f2-9c39-5e1603c2e061 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:02.475697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:02.476386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.476669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.476947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.477728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:02.478115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:02.479672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:02.503966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 381/772] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:02.507895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:02.535040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 392/773] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:02.536438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038a2de-ea17-4164-95a0-b3bbd97b40d2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:02.639812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:02.664631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052e0e-ddd1-4ca8-879e-9c0eb7174c7b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:02.677319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038a2de-ea17-4164-95a0-b3bbd97b40d2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:02.688905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:02.706471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176c87a-10b6-4592-b035-cbca20bd0957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:02.707540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c87a-10b6-4592-b035-cbca20bd0957 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.708104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c87a-10b6-4592-b035-cbca20bd0957 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.708382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c87a-10b6-4592-b035-cbca20bd0957 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.709256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176c87a-10b6-4592-b035-cbca20bd0957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:02.711384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176c87a-10b6-4592-b035-cbca20bd0957 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:02.718940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 382/774] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:02 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:02.719152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:02.719755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.720045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.720254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acf772-ac64-49ce-8c0a-deedf855230b None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.720356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.720542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acf772-ac64-49ce-8c0a-deedf855230b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.720818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acf772-ac64-49ce-8c0a-deedf855230b None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.721149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=4ee3b5df27184223a916ceccb0ee4717) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:02.721468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:02.722417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 383/775] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:02.723111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:02.728341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06608e8d-a1f5-4184-8780-1aa6ab333792 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:02.728656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06608e8d-a1f5-4184-8780-1aa6ab333792 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:02.728903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06608e8d-a1f5-4184-8780-1aa6ab333792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:02.732631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 393/776] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:02 2022] GET /identity/v3/role_assignments?user.id=4ee3b5df27184223a916ceccb0ee4717&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:02.752085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-06608e8d-a1f5-4184-8780-1aa6ab333792 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:02.753405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-06608e8d-a1f5-4184-8780-1aa6ab333792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:02.860953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 384/777] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:02.978780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:03.093250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6cdbf-5659-4cd3-bd19-ddb68774206e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:03.141259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:03.141913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.142195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.142471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] PATH_INFO: `/v3/services/network` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:03.143064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:03.144456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:03.155355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:03.157686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:55:03.159708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:03.163534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 394/778] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:02 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:03.176363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06608e8d-a1f5-4184-8780-1aa6ab333792 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:03.283367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:03.289763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:03.318491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06608e8d-a1f5-4184-8780-1aa6ab333792 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:03.364312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:03.364743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.365025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.365551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:03.366208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] RBAC: Adding query filter params (name=network) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:03.366515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:03.367934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:03.374732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 385/779] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:03.378924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:03.380190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:03.539467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b73e4-dd2a-42bb-9f7f-269960ff8da3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:03.545953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9ccd8-19a1-4a06-a4f6-2a7fa7ec3b86 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:03.588669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:03.589603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.589935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.590209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:03.590913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] RBAC: Adding query filter params (type=network) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:03.591247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:03.593338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:03.604657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 386/780] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:03.606760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:03.610813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:03.633554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:03.634106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.638766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.639040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:03.657737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:03.664173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:03.665127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:03.665950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:03.669387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 395/781] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:03.684179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:03.814609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae3204-c717-4a9c-8bd9-5fd54cc08e52 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:03.857536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:03.866457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70584a-904f-4713-b7ba-c482ee5caa59 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:03.911725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:03.912570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.912989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.913415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:03.914215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:03.914673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:03.916196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:03.917227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:03.918519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:03.919017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:03.919557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:03.920460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] RBAC: Adding query filter params (interface=public, service_id=99d97764248241be83e4994bc3118d1e, region_id=RegionOne) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:03.921043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:03.922168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:03.928410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 396/782] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:55:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:03.937243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 387/783] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:55:03 2022] GET /identity/v3/endpoints?service_id=99d97764248241be83e4994bc3118d1e&interface=public®ion_id=RegionOne => generated 194 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:03.938482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:03.938623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:04.012729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:04.051028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a87900-5886-4859-a465-f5920f3d0a7a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:04.062523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051d3a3-6b43-4406-b778-dfcd5f9f5342 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:04.091604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:04.092114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:04.092497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:04.092913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.093585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:04.095097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:04.101218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 388/784] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:03 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:04.114258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:04.115225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:04.115686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:04.116113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.142086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:04.143818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:04.172346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:04.173499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:04.174570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:04.174570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:04.174570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:04.180797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 397/785] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:04.182199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:04.354720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78822444-a77f-459f-ad8b-818f12500790 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:04.400215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:04.407041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:04.408026 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:04.408504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:04.408973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.409891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:04.410410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:04.411997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:04.431262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 389/786] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:04.437570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:04.461749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:04.601338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d8115-666f-4835-8b8e-425df6c4628e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:04.650236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:04.652106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:04.652654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:04.653218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] PATH_INFO: `/v3/users/swift` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.662280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:04.664336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:04.678093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:04.679461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:55:04.680649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:04.683632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 398/787] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:04.692281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:04.828477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca2ff5-f9e3-4d44-a56b-2ba6a0ea78e1 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:04.868236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d35ff-f8e1-4a95-9ec9-039898533611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:04.868918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d35ff-f8e1-4a95-9ec9-039898533611 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:04.869257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d35ff-f8e1-4a95-9ec9-039898533611 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:04.869540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d35ff-f8e1-4a95-9ec9-039898533611 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:04.871012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d35ff-f8e1-4a95-9ec9-039898533611 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:04.871337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d35ff-f8e1-4a95-9ec9-039898533611 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:04.873091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d35ff-f8e1-4a95-9ec9-039898533611 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:04.896447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 390/788] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Aug 29 18:55:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:04.902927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:05.009304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:05.020966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6586e-52ea-46b9-9477-8840424fce7f None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.021292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6586e-52ea-46b9-9477-8840424fce7f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.021535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6586e-52ea-46b9-9477-8840424fce7f None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.023238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 391/789] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:05.029639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f12bd4-a5bd-4c3e-8f51-03ac7b837e17 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.029925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f12bd4-a5bd-4c3e-8f51-03ac7b837e17 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.030190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f12bd4-a5bd-4c3e-8f51-03ac7b837e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.053613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e9f12bd4-a5bd-4c3e-8f51-03ac7b837e17 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:05.054705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80d8bc-a85e-4982-9c61-46b615a56129 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:05.054892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e9f12bd4-a5bd-4c3e-8f51-03ac7b837e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:05.114019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:05.114666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.114940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.115233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.145639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:05.147354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:05.154128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:05.177062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:05.178088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:05.179024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:05.179024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:05.179024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:05.180147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 399/790] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:05.186522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:05.205567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 392/791] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:05.358154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543058-035f-4dd5-b8f0-bfc2b8816a5e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:05.374383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6034e-9388-4e0b-8584-036badbe7d7d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.374905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6034e-9388-4e0b-8584-036badbe7d7d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.374983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6034e-9388-4e0b-8584-036badbe7d7d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.377821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 393/792] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:05.384088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e543-2cad-48c3-8c69-7652ffaa0a01 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.384367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e543-2cad-48c3-8c69-7652ffaa0a01 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.384640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e543-2cad-48c3-8c69-7652ffaa0a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.391798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:05.411680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e402e543-2cad-48c3-8c69-7652ffaa0a01 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:05.412884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e402e543-2cad-48c3-8c69-7652ffaa0a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:05.420503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:05.421141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.421426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.421705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.422367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:05.422693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:05.424109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:05.442184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 400/793] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:05.447936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:05.616796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2b1fb-e430-4f30-8f3c-d69d9c0bbc20 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:05.657602 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3d366-5277-4b9a-9d94-0f08a7f2bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:05.658553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d366-5277-4b9a-9d94-0f08a7f2bf89 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.659007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d366-5277-4b9a-9d94-0f08a7f2bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.659482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d366-5277-4b9a-9d94-0f08a7f2bf89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.671113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 394/794] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:05.671113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e543-2cad-48c3-8c69-7652ffaa0a01 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:05.677716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3d366-5277-4b9a-9d94-0f08a7f2bf89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:05.679277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3d366-5277-4b9a-9d94-0f08a7f2bf89 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-32a3d366-5277-4b9a-9d94-0f08a7f2bf89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:05.695279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:05.696258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:05.697596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 401/795] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:05.699728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4379-a3f5-4bfb-bd92-48190fa94e1e None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.700214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4379-a3f5-4bfb-bd92-48190fa94e1e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.700693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4379-a3f5-4bfb-bd92-48190fa94e1e None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.703802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 402/796] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:05.704913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4379-a3f5-4bfb-bd92-48190fa94e1e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:05.757045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:05.813967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e543-2cad-48c3-8c69-7652ffaa0a01 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:05.826120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:05.835261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4379-a3f5-4bfb-bd92-48190fa94e1e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:05.870265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e5ffb-aee9-4206-8f9c-6fab0e4350d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:05.872121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e5ffb-aee9-4206-8f9c-6fab0e4350d3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.872732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e5ffb-aee9-4206-8f9c-6fab0e4350d3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.873205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e5ffb-aee9-4206-8f9c-6fab0e4350d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.883324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:05.884121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.884510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.884926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.885827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:05.886331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:05.888189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:05.891532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e5ffb-aee9-4206-8f9c-6fab0e4350d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-ed2e5ffb-aee9-4206-8f9c-6fab0e4350d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:05.898465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:05.899464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:05.900314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:05.900953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 395/797] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:05.907427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d669-2558-4efe-91d5-6ff980a57cf1 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:05.907427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d669-2558-4efe-91d5-6ff980a57cf1 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:05.907427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d669-2558-4efe-91d5-6ff980a57cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:05.917914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 403/798] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:05.918916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:05.930916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a540d669-2558-4efe-91d5-6ff980a57cf1 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:05.933041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a540d669-2558-4efe-91d5-6ff980a57cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:06.028855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 396/799] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:06.029758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d669-2558-4efe-91d5-6ff980a57cf1 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.080046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f422-a8b6-4c72-ab4e-c59bf01fa383 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.082164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:06.132848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.133892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.134454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.134933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.136346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:06.136873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:06.138983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:06.154990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 404/800] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:06.156385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.156862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:06.159881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d669-2558-4efe-91d5-6ff980a57cf1 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.201335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.201701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.202083 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.202453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/7a0e57abdb8d4966a2066a85bbfbc15e/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.205036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=7a0e57abdb8d4966a2066a85bbfbc15e, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:06.207762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:06.231290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 397/801] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:55:06 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/7a0e57abdb8d4966a2066a85bbfbc15e/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:55:06.232560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.281973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db9c7a-227b-4de7-9cfa-1a5d982cd304 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.322150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.324089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.324875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.325571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.342395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:06.348870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:06.350171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:55:06.351234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:06.353683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 405/802] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:06.356935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.399596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655f13d-1fb5-4782-a003-007f203eeb37 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.411531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:06.460347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.461763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.462368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.463004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.478963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:06.480737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:06.500592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:06.501781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:06.502711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:06.502711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:06.502711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:06.502711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 398/803] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:06.502943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.523297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:06.532021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06936f60-42b0-4a36-b31b-7f48f96812ec None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.581550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.582187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.582463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.582832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.583925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:06.586410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:06.596859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:06.597943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:06.598787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 406/804] 10.222.0.94 () {56 vars in 1129 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:06.599616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.659894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7607-e145-45e5-ba60-3970e7dec874 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.720386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.721103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.721223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:06.721642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.721642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.722463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:06.722710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:06.724348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:06.734482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 399/805] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:06.735405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.742772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c59c4e-6b10-4361-90be-96a509ac3bd9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.784753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81299b-d785-4ef6-afd9-0100ff809be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.785677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81299b-d785-4ef6-afd9-0100ff809be1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.786142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81299b-d785-4ef6-afd9-0100ff809be1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.786624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81299b-d785-4ef6-afd9-0100ff809be1 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.787309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81299b-d785-4ef6-afd9-0100ff809be1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:06.787622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81299b-d785-4ef6-afd9-0100ff809be1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:06.789098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81299b-d785-4ef6-afd9-0100ff809be1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:06.810772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 407/806] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:06.812302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f49617-1bb2-4368-88e4-649530dd96d7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.812419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f49617-1bb2-4368-88e4-649530dd96d7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.812707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f49617-1bb2-4368-88e4-649530dd96d7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.814102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 408/807] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:06.814756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f49617-1bb2-4368-88e4-649530dd96d7 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.839587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:06.879176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd1454-f51d-4d50-8a7a-1ac034ed2813 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:06.928031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:06.928559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:06.928833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:06.929104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:06.929733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:06.930030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:06.931418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:06.936987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 400/808] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:06.937632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:06.952971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f49617-1bb2-4368-88e4-649530dd96d7 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.003419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fc669-b169-4546-9934-728206e3e23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.004208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fc669-b169-4546-9934-728206e3e23f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.004480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fc669-b169-4546-9934-728206e3e23f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.004745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fc669-b169-4546-9934-728206e3e23f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.016323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fc669-b169-4546-9934-728206e3e23f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.018847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fc669-b169-4546-9934-728206e3e23f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.027677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-610fc669-b169-4546-9934-728206e3e23f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:07.037845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:07.039054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:07.039983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:07.039983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:07.039983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:07.042728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 409/809] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:07.044066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1ca3d-206a-4f51-86d1-acf89909de1a None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.044351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1ca3d-206a-4f51-86d1-acf89909de1a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.044624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1ca3d-206a-4f51-86d1-acf89909de1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.067431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-49f1ca3d-206a-4f51-86d1-acf89909de1a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:07.069216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-49f1ca3d-206a-4f51-86d1-acf89909de1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:07.070960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62239344-ebb5-4066-a4d4-8d31a89f5939 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.112058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.113056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.113529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.114003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.125107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.127677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.133841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:07.146812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:07.148274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:07.149231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 401/810] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:55:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:07.149815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.167399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 410/811] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:07.168667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1ca3d-206a-4f51-86d1-acf89909de1a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.295423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ceb5b-7c6b-47c6-ad0d-ad70a65a2b5d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.299751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1ca3d-206a-4f51-86d1-acf89909de1a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.340645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:07.344897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.346110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.346678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.347228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.348611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:07.349141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.349981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.350822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.351348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.351594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.352023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.352882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:07.353381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.356183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.365988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 402/812] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:07.365988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.378111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 411/813] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:07.379749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.457632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:07.507162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584b02d-cdb5-4c97-a478-3b6c74743ddf None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.514944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bc09c-139c-4867-88b6-dd8241b2901e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.551631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.552562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.552998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.553463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.554701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:07.555249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.558365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.563388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.564016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.564299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.564576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.565256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.566677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.582322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 403/814] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:07.583638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.596844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 412/815] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:07.597596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.663409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:07.725664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d76fef-8f3b-4497-a04c-37b29cbf5ba0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.747535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4266ff-aed3-4683-828f-0c8f663ff244 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.753417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:07.771469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.772481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.772865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.773216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.782275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.783931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:07.799137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:07.800381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:07.801595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:07.803358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 404/816] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:07.805160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.815033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.815881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.816264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.816639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] PATH_INFO: `/v3/services/network` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.817354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:07.818840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:07.834004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:07.835329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 18:55:07.836370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:07.841405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 413/817] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:07.841405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:07.941928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abc8aa-75a4-4afb-8278-037a9e952881 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:07.997451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:07.998354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:07.998791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:07.999268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:07.999381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684153b-cac6-417b-82de-b7bd3323fe91 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:08.015856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.058925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:08.059813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.060270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.060693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.061578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:08.062037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.063723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.077176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.088362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 414/818] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:08.090089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.113143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.114960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.222184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ca58-c058-4023-bc8a-55ab99308f01 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:08.253705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 405/819] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:08.254930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.271127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:08.271744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.272013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.272281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.272914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] RBAC: Adding query filter params (name=network) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:08.273216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.274587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.285041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 415/820] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:08.286030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.323821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.370531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.407411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3bd7-4274-49b4-9deb-e873da6fd412 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:08.429085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3580bd-7ffc-4145-b865-4a9330a4a4ba None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:08.451156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:08.451810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.452056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.452326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.465188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.468259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.481595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c32a-38b3-445f-9d11-581bd8a21993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:08.482587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c32a-38b3-445f-9d11-581bd8a21993 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.483046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c32a-38b3-445f-9d11-581bd8a21993 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.483536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c32a-38b3-445f-9d11-581bd8a21993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:08.485687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:08.486736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:08.487876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:08.487876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:08.487876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:08.491435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 406/821] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:08.492309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.505720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c32a-38b3-445f-9d11-581bd8a21993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.509776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c32a-38b3-445f-9d11-581bd8a21993 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f1e7c32a-38b3-445f-9d11-581bd8a21993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:08.527483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:08.528872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:08.529983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:08.529983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:08.529983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:08.533430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 416/822] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:08.534729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9dc67-53ab-4a08-8257-855beeb0b93a None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.535025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9dc67-53ab-4a08-8257-855beeb0b93a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.535330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9dc67-53ab-4a08-8257-855beeb0b93a None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.536678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 417/823] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:08.537375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9dc67-53ab-4a08-8257-855beeb0b93a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.624981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04611fc6-be40-47db-9094-725448a5d9d6 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:08.646157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.659029 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.669287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:08.669497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9dc67-53ab-4a08-8257-855beeb0b93a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:08.669913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.670197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.670475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.671146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] RBAC: Adding query filter params (type=network) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:08.671455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.672837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.685354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 407/824] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:08.685354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.716524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59ad25-cffb-4a1c-9767-ade6321d3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:08.717132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59ad25-cffb-4a1c-9767-ade6321d3cbe None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.717410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59ad25-cffb-4a1c-9767-ade6321d3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.717675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59ad25-cffb-4a1c-9767-ade6321d3cbe None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.718316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59ad25-cffb-4a1c-9767-ade6321d3cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:08.718617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59ad25-cffb-4a1c-9767-ade6321d3cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.720010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59ad25-cffb-4a1c-9767-ade6321d3cbe None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.739150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 418/825] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:08.739150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7e7a0-5fe9-48c0-a827-73d7ae77e6dc None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.739150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7e7a0-5fe9-48c0-a827-73d7ae77e6dc None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.739150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7e7a0-5fe9-48c0-a827-73d7ae77e6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.759245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-77e7e7a0-5fe9-48c0-a827-73d7ae77e6dc None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:08.760505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-77e7e7a0-5fe9-48c0-a827-73d7ae77e6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:08.820388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcc5dd-014e-4aa7-86ba-62e65597b8de None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:08.852837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 419/826] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:08.853687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7e7a0-5fe9-48c0-a827-73d7ae77e6dc None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.879466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:08.880001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:08.880283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:08.880560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:08.881220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:08.881528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:08.882906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:08.904659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 408/827] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:08.905662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:08.946420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.972072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:08.983347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7e7a0-5fe9-48c0-a827-73d7ae77e6dc None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.023183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.023710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc572-2a10-4576-b78e-1e9503d6581b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.024388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.024872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.025348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.026848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.028630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.051902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 420/828] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:09.053214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.072032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.072715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.073587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.073982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.086455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.089582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:09.108077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:09.109047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:09.109943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 409/829] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:09.117633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.187864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113a223-67fe-433d-a95f-635a3eb9fbd5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.220941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:09.233902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.234960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.235493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.235983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.245050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.247164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:09.261666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:09.263025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:09.264084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:09.267226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 421/830] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:09.268567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.290629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd216c-1a62-43e7-be2e-b7f17b509436 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.311445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:09.344196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.345266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.345845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.346352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.347367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:09.347696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.349166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.369256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 410/831] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:09.371979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.433863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba9db3-4871-421b-9a39-3f894d215a75 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.482202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.482768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.483051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.483532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.495752 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:09.501359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:09.502334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 422/832] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:09.508716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.521465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3a13c-9b2f-40ad-b544-95c3b4c62f72 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.571933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.572555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.572833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.573678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.574801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:09.577971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:09.578097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.578443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.595237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 411/833] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:09.595237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.638485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:09.655122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade218e-47ea-4edd-8c8f-6fa60e7a62bd None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.693204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.693878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.694163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.694439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/4ee3b5df27184223a916ceccb0ee4717/roles/6328cebc97c34f12bc2c15bafa4ba1e0` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.714063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=4ee3b5df27184223a916ceccb0ee4717, role_id=6328cebc97c34f12bc2c15bafa4ba1e0)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.715774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.735757 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcd9a2-7aad-4c06-8e03-9a84b14ccac8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.747106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 423/834] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:55:09 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/4ee3b5df27184223a916ceccb0ee4717/roles/6328cebc97c34f12bc2c15bafa4ba1e0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:55:09.747106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.809460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5a4e9-9800-47f2-8351-08af0232fd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.810632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a4e9-9800-47f2-8351-08af0232fd2d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.811195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a4e9-9800-47f2-8351-08af0232fd2d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.811731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5a4e9-9800-47f2-8351-08af0232fd2d None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.813077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a4e9-9800-47f2-8351-08af0232fd2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:09.813639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a4e9-9800-47f2-8351-08af0232fd2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.816267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5a4e9-9800-47f2-8351-08af0232fd2d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.831821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 412/835] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:55:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:09.834320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc104a-9801-46d3-b77c-f5dded3e2684 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.835085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc104a-9801-46d3-b77c-f5dded3e2684 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.835648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc104a-9801-46d3-b77c-f5dded3e2684 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.838549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 413/836] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:09.839934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc104a-9801-46d3-b77c-f5dded3e2684 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:09.913863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:09.923114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7399c5-19bb-4902-ad4b-dc3c6cdc92a9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:09.977579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2c5f6-22e9-40a1-9bde-66f6aed9a7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:09.978195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c5f6-22e9-40a1-9bde-66f6aed9a7e2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.978476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c5f6-22e9-40a1-9bde-66f6aed9a7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.978741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2c5f6-22e9-40a1-9bde-66f6aed9a7e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.980049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c5f6-22e9-40a1-9bde-66f6aed9a7e2 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=8adead3860f144099efa11cefa83a37f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:09.980640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c5f6-22e9-40a1-9bde-66f6aed9a7e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:09.982085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2c5f6-22e9-40a1-9bde-66f6aed9a7e2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:09.991405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 424/837] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:09 2022] GET /identity/v3/role_assignments?user.id=8adead3860f144099efa11cefa83a37f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:09.991405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d6bd5-c45c-4b4a-8ffb-1e929db258de None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:09.991405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d6bd5-c45c-4b4a-8ffb-1e929db258de None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:09.991405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d6bd5-c45c-4b4a-8ffb-1e929db258de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:09.994587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:10.008366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc104a-9801-46d3-b77c-f5dded3e2684 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:10.015641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d37d6bd5-c45c-4b4a-8ffb-1e929db258de None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:10.017018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d37d6bd5-c45c-4b4a-8ffb-1e929db258de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:10.071331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:10.072784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.073938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.074568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.092294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:10.094044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:10.111003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:10.112149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:10.113082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:10.113082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:10.113082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:10.118954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 414/838] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:10.119188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 425/839] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:10.121099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.260143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95033a-7a7b-48a9-b505-49914bc16b19 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:10.305037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:10.306064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.306545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.307025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.308060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:10.308370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:10.309884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:10.323511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 415/840] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:10.329090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d6bd5-c45c-4b4a-8ffb-1e929db258de None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.392336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.407807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:10.454241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:10.465711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d6bd5-c45c-4b4a-8ffb-1e929db258de None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:10.535491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:10.536160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.536547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.536829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] PATH_INFO: `/v3/users/swift` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.548679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:10.553030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:10.557149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263efdc-1ea7-4f76-9250-c24b17b74059 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:10.593388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:10.594620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 18:55:10.595975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:10.600503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 426/841] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:10.605399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.613633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:10.614526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.614963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.615520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.616275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:10.618068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:10.637731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:10.638798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:10.644698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 416/842] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:10.645507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.746517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:10.817719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a73f4-81dd-4ffe-a7b7-70fd685a6641 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:10.852342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cbc19-6a89-46b9-811f-8a9bd2971500 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:10.909835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:10.910823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.911452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.911937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.913265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:10.913844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:10.916527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:10.927690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:10.928330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:10.928621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:10.929087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:10.929752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:10.930061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:10.933159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:10.943167 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 417/843] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:10.948524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.967456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 427/844] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:10.969745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:10.983724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:11.109464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3a65-bf6a-45ba-8cf7-6065158066b8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:11.120182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998a53-52d0-4e30-a11a-a8143cfb77e2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:11.150512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:11.151842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.152244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.152578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.153387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:11.153790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:11.155294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:11.157125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:11.167555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 418/845] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:11.179226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:11.179866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.180150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.180425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.286172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:11.288055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:11.300104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:11.411591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 428/846] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:55:11.418669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:11.566484 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a944a38-c152-4524-bd1c-65427ce006bd None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:11.605637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:11.606283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.606561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.606876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.619199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:11.621201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:11.640880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:11.641877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:11.642791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:11.642791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:11.642791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:11.642791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 419/847] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:11.648981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:11.779745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677624-f6fa-420a-bbbf-0d7de8d265f7 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:11.791326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:11.819675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1569d-bf77-4611-979c-8b6be12a82f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:11.820667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1569d-bf77-4611-979c-8b6be12a82f0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.821143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1569d-bf77-4611-979c-8b6be12a82f0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.821635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1569d-bf77-4611-979c-8b6be12a82f0 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.822476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1569d-bf77-4611-979c-8b6be12a82f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:11.823202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1569d-bf77-4611-979c-8b6be12a82f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:11.824916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1569d-bf77-4611-979c-8b6be12a82f0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:11.848192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 429/848] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:11.857092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:11.924460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:11.956557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8959d7-57cf-42bd-81c3-16fddb409439 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.957574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8959d7-57cf-42bd-81c3-16fddb409439 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.958065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8959d7-57cf-42bd-81c3-16fddb409439 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.960138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 430/849] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:11.966049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffd43e-1eac-45d4-8b16-b294edfddf20 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:11.966634 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffd43e-1eac-45d4-8b16-b294edfddf20 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:11.967134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffd43e-1eac-45d4-8b16-b294edfddf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:11.984256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe739be0-64f4-4f13-8e3b-c74c617a7d7b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:11.992681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e9ffd43e-1eac-45d4-8b16-b294edfddf20 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:11.994274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e9ffd43e-1eac-45d4-8b16-b294edfddf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:12.031356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:12.032076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.032358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.032636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.046521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:12.048027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:12.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:12.064901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:12.065783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:12.068283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 420/850] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:12.079107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:12.086239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 431/851] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:12.206832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca86d6b-3db1-4bc2-8ef2-4812fba28245 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.207304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca86d6b-3db1-4bc2-8ef2-4812fba28245 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.207680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca86d6b-3db1-4bc2-8ef2-4812fba28245 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.209416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 432/852] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:12.215756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1cd4-c671-4781-b5b3-8dba0676ee4a None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.216055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1cd4-c671-4781-b5b3-8dba0676ee4a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.216530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1cd4-c671-4781-b5b3-8dba0676ee4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.236073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c91d1cd4-c671-4781-b5b3-8dba0676ee4a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:12.237550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c91d1cd4-c671-4781-b5b3-8dba0676ee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:12.254274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0a0b8-e0fd-44a3-81c5-c4c65e239bf1 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:12.274161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:12.323381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:12.323381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.323381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.323381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.323821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:12.323821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:12.324653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 433/853] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:12.324926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:12.355330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 421/854] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:12.358394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1cd4-c671-4781-b5b3-8dba0676ee4a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:12.409483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:12.514711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1cd4-c671-4781-b5b3-8dba0676ee4a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:12.552605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:12.553567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.554089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.554553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.555505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=7a0e57abdb8d4966a2066a85bbfbc15e) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:12.555997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:12.557826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:12.565310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 434/855] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:12 2022] GET /identity/v3/role_assignments?user.id=7a0e57abdb8d4966a2066a85bbfbc15e&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:12.600426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:12.671376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:12.762891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bad91-60ef-4381-9bb7-d6740335faf6 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:12.820119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:12.823692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:12.824698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.825202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.825676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.838017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13985a46-7f2f-40a1-8e97-029062745007 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:12.844291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:12.850317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:12.851520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:12.852378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 422/856] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:55:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:12.860166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:12.890062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:12.891496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:12.892033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:12.892458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:12.893232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:12.894907 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:12.907932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:12.909114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:12.909995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 435/857] 10.222.0.94 () {56 vars in 1132 bytes} [Mon Aug 29 18:55:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:12.915783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:12.917502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.019545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed5875-116e-44a1-ab0e-265a19136736 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.070193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dac1b-490d-42af-9209-7b31523642b1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.074291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.075418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.075418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.075619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.076147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:13.076453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.077837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.087603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 423/858] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:55:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:13.093247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.127202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.128694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.129593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.130619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.131779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:13.132621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.135227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.151953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 436/859] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:13.152975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.178861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:13.225380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:13.243658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afa126-32ee-432a-8285-9dcbea1f9953 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.269416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d02d0-24ea-4f89-93ae-902d80e66bae None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.302465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b21aa9-ee64-434b-9048-82c9f1805736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.303631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b21aa9-ee64-434b-9048-82c9f1805736 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.304127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b21aa9-ee64-434b-9048-82c9f1805736 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.304663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b21aa9-ee64-434b-9048-82c9f1805736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.306653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.307939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.308427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.308817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.309513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:13.309996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.311414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.319977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b21aa9-ee64-434b-9048-82c9f1805736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.321893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b21aa9-ee64-434b-9048-82c9f1805736 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.324096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 437/860] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:13 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-03b21aa9-ee64-434b-9048-82c9f1805736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:13.345291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:13.347015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:13.348761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:13.348761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:13.348761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:13.353025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 424/861] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:13.359236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.390468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd84f8-b0d6-4c5e-b31f-c13a01d010e9 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.390699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd84f8-b0d6-4c5e-b31f-c13a01d010e9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.391028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd84f8-b0d6-4c5e-b31f-c13a01d010e9 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.392633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 425/862] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:13.398215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa735be-62ef-489c-94cf-20196af42d74 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.398566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa735be-62ef-489c-94cf-20196af42d74 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.398846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa735be-62ef-489c-94cf-20196af42d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.419307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1fa735be-62ef-489c-94cf-20196af42d74 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:13.420647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1fa735be-62ef-489c-94cf-20196af42d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:13.464367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:13.485402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de0f0b-f703-476d-9b4b-952003fce658 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.531755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 426/863] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:13.539264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.540143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.540609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.540881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.541524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:13.541861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.543259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.557316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 438/864] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:13.564477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa735be-62ef-489c-94cf-20196af42d74 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.657053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:13.687790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa735be-62ef-489c-94cf-20196af42d74 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.726131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.727055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.727509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.727951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.737188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.738889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:13.753480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:13.754675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:13.755899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:13.758200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 427/865] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:55:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:13.764907 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.820784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.899526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a64bc0-acef-4812-b6fb-8c1838844767 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.946886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:13.947548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:13.948720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:13.949202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:13.950216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:13.950775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:13.953106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:13.973758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fdd2f-9765-493d-bb7f-d205939f0180 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:13.976952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 439/866] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Aug 29 18:55:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:13.985986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:13.985986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:14.023158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:14.023780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.024054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.024319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.024980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:14.027056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:14.031356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:14.046998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 428/867] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:55:13 2022] POST /identity/v3/services => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:55:14.132231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c4586-58b0-4c58-84f4-6b96c1c3f2c6 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:14.186188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:14.186827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.187142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.187455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.266949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:14.271872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:14.273473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:14.400549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 440/868] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:13 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:14.407161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:14.537227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95991d-1914-4015-a38a-ce0166befdd5 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:14.576373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fdd5b-8a1e-4f0f-abb9-8478d132ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:14.577026 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fdd5b-8a1e-4f0f-abb9-8478d132ce1a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.577306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fdd5b-8a1e-4f0f-abb9-8478d132ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.577581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fdd5b-8a1e-4f0f-abb9-8478d132ce1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.590154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fdd5b-8a1e-4f0f-abb9-8478d132ce1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:14.591936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fdd5b-8a1e-4f0f-abb9-8478d132ce1a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-f14fdd5b-8a1e-4f0f-abb9-8478d132ce1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:14.609047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:14.610059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:14.611129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:14.611129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:14.611129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:14.615430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 429/869] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:14.616538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:14.739499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:14.745942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a307b6c-98bc-42f5-a529-b20575a4ac90 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:14.760493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b21e-9552-4cda-b3ed-e8a9478ef909 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.761298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b21e-9552-4cda-b3ed-e8a9478ef909 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.761298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b21e-9552-4cda-b3ed-e8a9478ef909 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.763111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 430/870] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:14.771718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848508c7-17ef-4570-8ac4-e75aff120bc5 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.772572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848508c7-17ef-4570-8ac4-e75aff120bc5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.773340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848508c7-17ef-4570-8ac4-e75aff120bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.792574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:14.793193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:14.793486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:14.793764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:14.794416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:14.794722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:14.796157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:14.797633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-848508c7-17ef-4570-8ac4-e75aff120bc5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:14.799138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-848508c7-17ef-4570-8ac4-e75aff120bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:14.815251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 441/871] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:14.818797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:14.865124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:14.907116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 431/872] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:14.981219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d116cc0-1db4-40f4-a619-94d996c90ae5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:15.018230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:15.019849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.020657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.021454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.031002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:15.033020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:15.047223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:15.049068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:15.050984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:15.054522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 442/873] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:15.061128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848508c7-17ef-4570-8ac4-e75aff120bc5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:15.188259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848508c7-17ef-4570-8ac4-e75aff120bc5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:15.212732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:15.238431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:15.238743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:15.240372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.241007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.241508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.242834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:15.243397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:15.247131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:15.273342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 432/874] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:15.279878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:15.335889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:15.353581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0d1cc-d8a6-45e2-ae82-d5b0f53499e1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:15.391725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:15.392774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.393265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.393737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.394536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:15.396166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:15.406804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:15.408082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:15.408932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:15.416428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 443/875] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:15.419143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:15.443688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0564752-b9f0-4375-a5e6-3f01cc169a76 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:15.488996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7edf9d-6422-4563-b4e4-20c8f8ab418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:15.489614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7edf9d-6422-4563-b4e4-20c8f8ab418e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.489890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7edf9d-6422-4563-b4e4-20c8f8ab418e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.490157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7edf9d-6422-4563-b4e4-20c8f8ab418e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.490958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7edf9d-6422-4563-b4e4-20c8f8ab418e None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=4ee3b5df27184223a916ceccb0ee4717) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:15.491311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7edf9d-6422-4563-b4e4-20c8f8ab418e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:15.492795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7edf9d-6422-4563-b4e4-20c8f8ab418e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:15.507424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 433/876] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:15 2022] GET /identity/v3/role_assignments?user.id=4ee3b5df27184223a916ceccb0ee4717&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:15.530759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:15.562803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e02b59-c2f0-462e-a8c2-4c69b5c64157 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:15.605460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:15.606348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.606888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.607245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.608224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] RBAC: Adding query filter params (name=object-store) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:15.608568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:15.609722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c983c-91d2-4df5-8a02-de340dbd32e4 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.609971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:15.610210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c983c-91d2-4df5-8a02-de340dbd32e4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.610695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c983c-91d2-4df5-8a02-de340dbd32e4 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.613172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 434/877] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:15.619372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bfdd4-d1ec-46d9-8b22-bea4befdfa96 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.620768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bfdd4-d1ec-46d9-8b22-bea4befdfa96 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.621477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bfdd4-d1ec-46d9-8b22-bea4befdfa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.623147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 444/878] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:15.633197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:15.659958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-917bfdd4-d1ec-46d9-8b22-bea4befdfa96 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:15.662576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-917bfdd4-d1ec-46d9-8b22-bea4befdfa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:15.788341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 435/879] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:15.820759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756dbfb-eabc-44fa-92b2-27233ac799ce None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:15.864750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70ef2f-3c4a-49da-9411-bbb144008ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:15.865650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ef2f-3c4a-49da-9411-bbb144008ea3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:15.866128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ef2f-3c4a-49da-9411-bbb144008ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:15.866586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ef2f-3c4a-49da-9411-bbb144008ea3 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:15.867756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ef2f-3c4a-49da-9411-bbb144008ea3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:15.868307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ef2f-3c4a-49da-9411-bbb144008ea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:15.871617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ef2f-3c4a-49da-9411-bbb144008ea3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:15.878226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 445/880] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:16.086755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bfdd4-d1ec-46d9-8b22-bea4befdfa96 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:16.136354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:16.224223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bfdd4-d1ec-46d9-8b22-bea4befdfa96 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:16.264683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:16.265660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:16.266191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:16.266679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:16.267604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:16.269597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:16.270741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8474-fed6-46da-b26b-dcc483ba4fd8 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:16.271228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8474-fed6-46da-b26b-dcc483ba4fd8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:16.271497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8474-fed6-46da-b26b-dcc483ba4fd8 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:16.273296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 446/881] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:16.281795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa29875-6e7c-402e-887e-fe15a6f5d1c2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:16.282149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa29875-6e7c-402e-887e-fe15a6f5d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:16.282434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa29875-6e7c-402e-887e-fe15a6f5d1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:16.291965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 436/882] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:55:16 2022] POST /identity/v3/services => generated 269 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:16.305803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-caa29875-6e7c-402e-887e-fe15a6f5d1c2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:16.307148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-caa29875-6e7c-402e-887e-fe15a6f5d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:16.320881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:16.401998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 447/883] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:16.695489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:16.814101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:16.827487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43195ac0-f719-495a-ba37-0b2a46f5a119 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:16.865564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:16.866225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:16.866506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:16.866780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:16.867425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:16.868915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:16.879890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:16.880949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:16.882016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 437/884] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:16.888259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa29875-6e7c-402e-887e-fe15a6f5d1c2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:17.004868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa29875-6e7c-402e-887e-fe15a6f5d1c2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:17.051160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:17.052455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.053116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.053564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.054436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:17.054887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:17.056431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:17.059401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:17.063785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 448/885] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:17.069165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:17.201354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e94a-2f82-4e45-b8d5-0de1c91abc74 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:17.239783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:17.240435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.240723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.241025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.241715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:17.242025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:17.243553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:17.249621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 438/886] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:17.255598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:17.376025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5afb9-c087-491d-81df-432198918d9a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:17.423770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574a74f-ace2-46b9-bb69-846d551fbaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:17.424682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574a74f-ace2-46b9-bb69-846d551fbaeb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.425118 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574a74f-ace2-46b9-bb69-846d551fbaeb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.425546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574a74f-ace2-46b9-bb69-846d551fbaeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.426543 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574a74f-ace2-46b9-bb69-846d551fbaeb None admin] RBAC: Adding query filter params (interface=public, service_id=7b86d78d1f864beb85f80781ede80edf, region_id=RegionOne) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:17.426999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574a74f-ace2-46b9-bb69-846d551fbaeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:17.428545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574a74f-ace2-46b9-bb69-846d551fbaeb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:17.438017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 449/887] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:55:17 2022] GET /identity/v3/endpoints?service_id=7b86d78d1f864beb85f80781ede80edf&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:17.443266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:17.472331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:17.577660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e744-8422-4b8b-a942-f1f8f19b396b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:17.624950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60b4d6-b294-49c5-89f2-138809809887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:17.625574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60b4d6-b294-49c5-89f2-138809809887 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.625860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60b4d6-b294-49c5-89f2-138809809887 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.626194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60b4d6-b294-49c5-89f2-138809809887 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.632150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60b4d6-b294-49c5-89f2-138809809887 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:17.633602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60b4d6-b294-49c5-89f2-138809809887 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:17.640883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 439/888] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:17 2022] GET /identity/v3/services => generated 1268 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:55:17.845020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88636982-56b0-4e91-aa68-66201e5c8b04 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.845931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88636982-56b0-4e91-aa68-66201e5c8b04 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.846660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88636982-56b0-4e91-aa68-66201e5c8b04 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.849942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 450/889] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:17.855786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba79b5-c349-48d0-8a4c-588b1959bd20 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:17.856298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba79b5-c349-48d0-8a4c-588b1959bd20 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:17.856398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba79b5-c349-48d0-8a4c-588b1959bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:17.875344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d2ba79b5-c349-48d0-8a4c-588b1959bd20 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:17.876576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d2ba79b5-c349-48d0-8a4c-588b1959bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:17.955217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 440/890] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:18.097788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb778f-0d89-44b9-b71b-b6c6c187f42d None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.097982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb778f-0d89-44b9-b71b-b6c6c187f42d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.098254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb778f-0d89-44b9-b71b-b6c6c187f42d None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.099944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 451/891] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:18.105442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead0444-f13b-4c88-b047-e0981cb53d48 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.105821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead0444-f13b-4c88-b047-e0981cb53d48 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.106041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead0444-f13b-4c88-b047-e0981cb53d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.126313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-fead0444-f13b-4c88-b047-e0981cb53d48 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:18.128400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fead0444-f13b-4c88-b047-e0981cb53d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:18.141543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:18.225584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 441/892] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:18.249234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb778f-0d89-44b9-b71b-b6c6c187f42d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:18.338036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:18.390358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb778f-0d89-44b9-b71b-b6c6c187f42d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:18.429420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07105162-79a2-4ba9-a2f5-b1f4215612de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:18.430171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07105162-79a2-4ba9-a2f5-b1f4215612de None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.430456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07105162-79a2-4ba9-a2f5-b1f4215612de None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.430726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07105162-79a2-4ba9-a2f5-b1f4215612de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.441667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07105162-79a2-4ba9-a2f5-b1f4215612de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-07105162-79a2-4ba9-a2f5-b1f4215612de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:18.446590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:18.447669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:18.449281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 452/893] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Aug 29 18:55:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:18.455367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead0444-f13b-4c88-b047-e0981cb53d48 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:18.496192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7d2e-9309-4c87-a989-b2b02acb4813 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.496685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7d2e-9309-4c87-a989-b2b02acb4813 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.497155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7d2e-9309-4c87-a989-b2b02acb4813 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.500887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 453/894] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:18.507004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61ff9-98ba-4a15-ad81-fd13e81904b6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.507531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61ff9-98ba-4a15-ad81-fd13e81904b6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.507957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61ff9-98ba-4a15-ad81-fd13e81904b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.530089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7ad61ff9-98ba-4a15-ad81-fd13e81904b6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:18.531345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7ad61ff9-98ba-4a15-ad81-fd13e81904b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:18.603181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead0444-f13b-4c88-b047-e0981cb53d48 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:18.628591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 454/895] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:18.635476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad61ff9-98ba-4a15-ad81-fd13e81904b6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:18.650917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:18.651511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.651915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.652345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.653037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:18.653340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:18.656306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:18.679119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 442/896] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Aug 29 18:55:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:18.683960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:18.690305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:18.735105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:18.822593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad61ff9-98ba-4a15-ad81-fd13e81904b6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:18.855315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372d275-98ee-43ab-a31d-6347e4f9be9c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:18.880035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:18.880955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.881805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.882459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.884027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:18.886560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:18.899185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:18.900579 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:18.901582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:18.902041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:18.908237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 455/897] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:55:18 2022] POST /identity/v3/services => generated 241 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:18.909402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:18.927532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:18.929779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:18.953845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:18.954958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:18.956060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:18.956060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:18.956060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:18.959310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 443/898] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:18.965705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.051206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.062947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c249d-19df-476e-8d97-f57dae665253 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.077155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.106887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.106887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.106887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.106887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.107388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.108492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:19.119655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:19.121199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:19.122073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 456/899] 10.222.0.94 () {56 vars in 1132 bytes} [Mon Aug 29 18:55:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:19.128544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.129441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefd132-b1c3-409a-9ae7-afa21cb81e2f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.204562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.205110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.205418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.206211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.207960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:19.208874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.211467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.228522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 444/900] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:19.234745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.281026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d74210-d12d-4c89-8f38-b59df27c48f3 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.333394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.334414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.334900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.335391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.336235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:19.336767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.338447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.348040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 457/901] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:19.354824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.376292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade24d62-047a-4809-9093-4920274c6d60 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.398415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.421651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.423603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.424605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.425111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.425625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.434863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.436796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:19.453264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:19.454285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:19.455310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:19.462098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 445/902] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:19.463141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.502390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960ed49-528d-4db1-9a34-926b3f7d2301 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.543538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.544519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.545195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.545790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.546621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:19.547147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.548891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.555276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 458/903] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:19.562199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.608942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd2ac5-aa15-429e-be03-0898cfea01c6 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.649264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.650332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.650817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.652045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.653091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:19.653632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.655507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.678597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 446/904] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:19.684766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.694128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792cd8f-a802-4200-a47e-e4319d3a44de None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.696196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.731982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:19.734985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.735961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.736493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.736971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.737882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] RBAC: Adding query filter params (interface=public, service_id=1d5782336a354314b47fe2d20557d8ba, region_id=RegionOne) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:19.738393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.739996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.751199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 459/905] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/endpoints?service_id=1d5782336a354314b47fe2d20557d8ba&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:19.752856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:19.812258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9f6f-f5a7-46e4-bb7b-6f7248f86797 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.863230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.864085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.864207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.864388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.882227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a78aaa-6dc4-477c-9d06-596d373f4418 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:19.930550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909692a9-d277-497b-936e-2fbf2378e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:19.931203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909692a9-d277-497b-936e-2fbf2378e4cf None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.931654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909692a9-d277-497b-936e-2fbf2378e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.931926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909692a9-d277-497b-936e-2fbf2378e4cf None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.933240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909692a9-d277-497b-936e-2fbf2378e4cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.934822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909692a9-d277-497b-936e-2fbf2378e4cf None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.942100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 460/906] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 18:55:19.943553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d85f8-2d05-4722-b0a5-988c461b9314 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.943832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d85f8-2d05-4722-b0a5-988c461b9314 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.944166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d85f8-2d05-4722-b0a5-988c461b9314 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.947113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 461/907] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:19.953021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc9da1-22fd-4add-a238-2bab56f52d2b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:19.953605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc9da1-22fd-4add-a238-2bab56f52d2b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:19.954021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc9da1-22fd-4add-a238-2bab56f52d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:19.962958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:19.965044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:19.978433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-16dc9da1-22fd-4add-a238-2bab56f52d2b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:19.981587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-16dc9da1-22fd-4add-a238-2bab56f52d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:20.012359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:20.021301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:20.087326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 462/908] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:20.117778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 447/909] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:19 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:20.124073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc9da1-22fd-4add-a238-2bab56f52d2b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.272656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc9da1-22fd-4add-a238-2bab56f52d2b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:20.313345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:20.313986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:20.314272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:20.314538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:20.327237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:20.328725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:20.345598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:20.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:20.347550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:20.347550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:20.347550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:20.347550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 463/910] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:20.354293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.374482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.392388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:20.490631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfce51-284f-44da-92e9-23d98e0ba7de None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:20.492974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7078e7-b53a-4af1-966b-f61329c1ff30 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:20.534495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:20.535647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:20.535906 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:20.536332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:20.537324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:20.539802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:20.551121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:20.552775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:20.553536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:20.554255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:20.555660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:20.556416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:20.559036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:20.561205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:20.562156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 464/911] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:20.563746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:20.569461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.572056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:20.583661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 448/912] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:20.584639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.684044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:20.706729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d9c3-642b-48c9-82c7-83e342fda8d2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:20.723450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50148-6fb8-4758-9bb3-ba43b310769a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:20.753969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:20.755339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:20.755922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:20.756449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:20.757492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:20.757974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:20.760255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:20.768091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 465/913] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:20.775126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.781326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:20.782362 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:20.782859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:20.795185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:20.795367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:20.796981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:20.817941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:20.818985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:20.819942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:20.824929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 449/914] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:20.826078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.896574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:20.904992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e456c5f-0daa-48e5-8b01-36afaf2ff4ff None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:20.948782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:20.949406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:20.949684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:20.949957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:20.950606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:20.950909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:20.952011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7271a-c432-479c-a629-7b8ee8f2c521 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:20.952334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:20.958428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 466/915] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Aug 29 18:55:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:20.964127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:20.968399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:21.000000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:21.001260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.001720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.001997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.002719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:21.003039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:21.004552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:21.027481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 450/916] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:21.029229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:21.124542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14a687-e350-489d-9c7f-28591a9e0602 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:21.149969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13309b-cef7-43dc-b77b-36481e5cc7da None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:21.169241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:21.176453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23f452-ec80-45fb-ba0a-82338a45d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:21.177069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23f452-ec80-45fb-ba0a-82338a45d1a8 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.177344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23f452-ec80-45fb-ba0a-82338a45d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.177617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23f452-ec80-45fb-ba0a-82338a45d1a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.178288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23f452-ec80-45fb-ba0a-82338a45d1a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:21.179815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23f452-ec80-45fb-ba0a-82338a45d1a8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:21.196599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706e377-479e-4dcd-80b7-8e742f084dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:21.197449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e377-479e-4dcd-80b7-8e742f084dec None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.197955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e377-479e-4dcd-80b7-8e742f084dec None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.198917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e377-479e-4dcd-80b7-8e742f084dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.199802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706e377-479e-4dcd-80b7-8e742f084dec None admin] RBAC: Adding query filter params (role.id=6752b97865d54ef3a53d54dbadc24ae4, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=4ee3b5df27184223a916ceccb0ee4717) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:21.200263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706e377-479e-4dcd-80b7-8e742f084dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:21.202155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706e377-479e-4dcd-80b7-8e742f084dec None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:21.207443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 467/917] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:21.208998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fb419-88f5-4e54-b994-9cd0b0178eb0 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.209505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fb419-88f5-4e54-b994-9cd0b0178eb0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.219110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 451/918] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:21 2022] GET /identity/v3/role_assignments?user.id=4ee3b5df27184223a916ceccb0ee4717&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6752b97865d54ef3a53d54dbadc24ae4 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:21.219345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fb419-88f5-4e54-b994-9cd0b0178eb0 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.219345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 468/919] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:21.225067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d4c43-4760-4bcb-986a-a28840c94ef2 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.225483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d4c43-4760-4bcb-986a-a28840c94ef2 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.225835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d4c43-4760-4bcb-986a-a28840c94ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.248201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5b3d4c43-4760-4bcb-986a-a28840c94ef2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:21.249836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5b3d4c43-4760-4bcb-986a-a28840c94ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:21.346006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 452/920] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:21.644805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fb419-88f5-4e54-b994-9cd0b0178eb0 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:21.715006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:21.763674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fb419-88f5-4e54-b994-9cd0b0178eb0 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:21.807150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:21.808343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:21.808807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:21.809284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:21.810329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:21.812755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:21.823428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:21.824787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:21.826557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 469/921] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:21.832259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d4c43-4760-4bcb-986a-a28840c94ef2 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:21.959535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d4c43-4760-4bcb-986a-a28840c94ef2 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:21.964347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:22.002001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:22.002794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.002975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.003282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.003943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:22.004244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:22.005634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:22.013213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 453/922] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:22.019180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:22.132065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dae030-8344-4a0f-b916-dbd3e7b45895 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:22.159989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:22.172469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a045-853d-45be-93dd-63b99cbf6762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:22.173242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a045-853d-45be-93dd-63b99cbf6762 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.173624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a045-853d-45be-93dd-63b99cbf6762 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.173989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a045-853d-45be-93dd-63b99cbf6762 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.174789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95a045-853d-45be-93dd-63b99cbf6762 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:22.175219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95a045-853d-45be-93dd-63b99cbf6762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:22.176686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95a045-853d-45be-93dd-63b99cbf6762 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:22.183243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 470/923] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:22.190194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:22.210905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a09a-5e9f-4bcb-a5fa-4c589feb3963 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.211225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a09a-5e9f-4bcb-a5fa-4c589feb3963 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.211525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a09a-5e9f-4bcb-a5fa-4c589feb3963 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.213342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 471/924] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:22.220027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fedac7-8141-4e22-a424-8c6c7e7096d8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.220315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fedac7-8141-4e22-a424-8c6c7e7096d8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.220596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fedac7-8141-4e22-a424-8c6c7e7096d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.242856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-75fedac7-8141-4e22-a424-8c6c7e7096d8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:22.244113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-75fedac7-8141-4e22-a424-8c6c7e7096d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:22.318486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a50e7a-824e-4f24-9c14-f3ae05efec86 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:22.339209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 472/925] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:22.365404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:22.366028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.366308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.366576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.367318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] RBAC: Adding query filter params (interface=public, service_id=4732d95f975045028d267e4a9a66d016, region_id=RegionOne) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:22.367626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:22.369313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:22.378226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 454/926] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:55:22 2022] GET /identity/v3/endpoints?service_id=4732d95f975045028d267e4a9a66d016&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:22.383848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fedac7-8141-4e22-a424-8c6c7e7096d8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:22.516858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fedac7-8141-4e22-a424-8c6c7e7096d8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:22.542260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:22.562966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:22.563658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.563938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.564231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.564876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:22.566336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:22.574337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 473/927] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:22 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:55:22.682858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:22.723656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:22.822797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7fc1-afac-43f7-a6a3-e05d67f5cd9a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:22.862500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:22.863187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:22.863470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:22.863745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:22.864354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:22.865737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:22.876511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:22.877578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:22.878460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 455/928] 10.222.0.94 () {56 vars in 1132 bytes} [Mon Aug 29 18:55:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:22.884005 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:23.001888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4def-65ce-4582-9c41-eb8a3b860109 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:23.038937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:23.039643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.039958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.040420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.041065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:23.041374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:23.043643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:23.051209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 474/929] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:23.057267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:23.176711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:23.187124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed7167-2300-401e-ae90-34c1f944c82b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:23.231874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40526c88-bef8-41f4-b7eb-02ab73057676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:23.232710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526c88-bef8-41f4-b7eb-02ab73057676 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.233132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526c88-bef8-41f4-b7eb-02ab73057676 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.233483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526c88-bef8-41f4-b7eb-02ab73057676 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.234355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40526c88-bef8-41f4-b7eb-02ab73057676 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:23.235197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40526c88-bef8-41f4-b7eb-02ab73057676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:23.236653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40526c88-bef8-41f4-b7eb-02ab73057676 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:23.243831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 456/930] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:23.252548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:23.305687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:23.351214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df8a43-fb60-4ec7-b255-363e682cd4fd None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.351483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df8a43-fb60-4ec7-b255-363e682cd4fd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.351771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df8a43-fb60-4ec7-b255-363e682cd4fd None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.353415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 457/931] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:23.360148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a02c-5ab9-4018-aa7e-9569f669da8f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.360527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a02c-5ab9-4018-aa7e-9569f669da8f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.360803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a02c-5ab9-4018-aa7e-9569f669da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.367502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dc2a6-20a8-445f-ae58-70b837cb711c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:23.381885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a042a02c-5ab9-4018-aa7e-9569f669da8f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:23.383228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a042a02c-5ab9-4018-aa7e-9569f669da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:23.409316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e655f-12dc-4007-b419-79280d4c74fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:23.409955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e655f-12dc-4007-b419-79280d4c74fd None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.410240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e655f-12dc-4007-b419-79280d4c74fd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.410519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e655f-12dc-4007-b419-79280d4c74fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.411399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e655f-12dc-4007-b419-79280d4c74fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:23.412691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e655f-12dc-4007-b419-79280d4c74fd None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:23.439601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 475/932] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:23.455772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e1cb3-4732-4db0-b2ae-4694d5aa4fe5 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.456040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e1cb3-4732-4db0-b2ae-4694d5aa4fe5 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.456356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e1cb3-4732-4db0-b2ae-4694d5aa4fe5 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.459358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 476/933] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:23.464814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ec6f-1fbf-4840-8e6a-fcbf17ed6c3e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.465173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ec6f-1fbf-4840-8e6a-fcbf17ed6c3e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.465385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ec6f-1fbf-4840-8e6a-fcbf17ed6c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.487705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2ce6ec6f-1fbf-4840-8e6a-fcbf17ed6c3e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:23.488926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2ce6ec6f-1fbf-4840-8e6a-fcbf17ed6c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:23.497594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 458/934] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:23.590407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 477/935] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:23.784440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042a02c-5ab9-4018-aa7e-9569f669da8f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:23.882365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ec6f-1fbf-4840-8e6a-fcbf17ed6c3e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:23.903578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:23.922803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042a02c-5ab9-4018-aa7e-9569f669da8f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:23.925950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:23.961679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:23.962348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:23.962623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:23.962891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:23.980868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:23.986603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:23.987721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:23.988728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 459/936] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Aug 29 18:55:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:23.997674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:24.026489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ec6f-1fbf-4840-8e6a-fcbf17ed6c3e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:24.081748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:24.082352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.082637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.082892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.083622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:24.085016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:24.087687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING keystone.server.flask.common [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 18:55:24.109101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.sql.core [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:55:24.109101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 18:55:24.109101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [parameters: {'id': '19a47020d755477b8f98321138b58632', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 18:55:24.109101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=76733) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:55:24.124827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c415-031e-400a-9b60-549903d3992a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19a47020d755477b8f98321138b58632.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19a47020d755477b8f98321138b58632. Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:55:24.126716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result.read() Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 18:55:24.127627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application yield resource Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise self.value Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application session.commit() Aug 29 18:55:24.128640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:55:24.129531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:55:24.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result.read() Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:55:24.131909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [parameters: {'id': '19a47020d755477b8f98321138b58632', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:24.132784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:24.134993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19a47020d755477b8f98321138b58632. Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 478/937] 10.222.0.94 () {60 vars in 1140 bytes} [Mon Aug 29 18:55:23 2022] POST /identity/v3/projects => generated 262 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:24.138559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:24.212349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:24.213861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.214626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.215464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.216281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:24.217839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:24.219592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:24.238950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 460/938] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Aug 29 18:55:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:24.241907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:24.263690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:24.287727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:24.304438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339154e-3df6-42b3-bf9c-d715d50787c5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:24.344749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:24.345787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.346290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.346766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.355376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:24.357539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:24.370145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:24.371323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:24.372427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.374462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 479/939] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:24.380084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:24.382571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458d00f-5777-42de-a7df-7b1567ca2099 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:24.421019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:24.421686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.421956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.422232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.434570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:24.436158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:24.452634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:24.454390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:24.456135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:24.456135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:24.456135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:24.458577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 461/940] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:24.459629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:24.511792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c7be-0ae3-4f87-9944-941347df0e58 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:24.527043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:24.574284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:24.575281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.575706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ecf4a-3c65-4be2-9553-acb2ad2f56fb None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:24.576167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.576619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.577732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:24.578336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:24.580154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:24.611502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 480/941] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:24.620486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff4fc9-b217-49b9-9a02-c5cb3f722fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:24.621125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4fc9-b217-49b9-9a02-c5cb3f722fce None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.621409 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4fc9-b217-49b9-9a02-c5cb3f722fce None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.621684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4fc9-b217-49b9-9a02-c5cb3f722fce None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.622362 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff4fc9-b217-49b9-9a02-c5cb3f722fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:24.622669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff4fc9-b217-49b9-9a02-c5cb3f722fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:24.624609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff4fc9-b217-49b9-9a02-c5cb3f722fce None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:24.640658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 462/942] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:24.646962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:24.670949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:24.794679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ed91-b897-41c2-9924-2455ca9c7028 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:24.826160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423941-f07a-4f36-8dab-c44c5a090ad8 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.826735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423941-f07a-4f36-8dab-c44c5a090ad8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.827011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6423941-f07a-4f36-8dab-c44c5a090ad8 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.828570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 463/943] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:24.834229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc84e4-f75f-47c4-869b-933b359f805e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.834975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc84e4-f75f-47c4-869b-933b359f805e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.836693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc84e4-f75f-47c4-869b-933b359f805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.849094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:24.849747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:24.850026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:24.850298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:24.861728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d1fc84e4-f75f-47c4-869b-933b359f805e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:24.862884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d1fc84e4-f75f-47c4-869b-933b359f805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:24.864028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:24.866084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:24.883843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:24.884860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:24.886614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 481/944] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:55:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:24.891553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:24.965502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 464/945] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:24.990299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:25.029468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f74b-7e17-42e5-9282-dd82f2d68f8f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.067535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.068149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.068423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.068690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.069416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:25.069715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.071161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.091860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 482/946] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Aug 29 18:55:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:25.098182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc84e4-f75f-47c4-869b-933b359f805e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.165345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:25.224020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc84e4-f75f-47c4-869b-933b359f805e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.261495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.262248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.262536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.262813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.274747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.277150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:25.295187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:25.296200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:25.297064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:25.297064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:25.297064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:25.297064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 465/947] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:25.303427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.312814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.434381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8092385-3951-45b5-8c52-633de0804426 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.440816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb32e-4c43-4183-b8e2-00efc010ddc2 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.443008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:25.466761 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:25.478972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.480716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.481231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.481776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.482907 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:25.483446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.483747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.484388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.484667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.484943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.485534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.486084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.486886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:25.497217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:25.498315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:25.504206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 466/948] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:25.505088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.513510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 483/949] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:25.516142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.657486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aed85-abf9-4e2c-93ee-a0d56da4fcc9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.662359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59c711-8463-47f9-82da-e6a345fc9683 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.697581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d570fc-03da-4ddd-96cd-b9e37ba9ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.698660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d570fc-03da-4ddd-96cd-b9e37ba9ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.699100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d570fc-03da-4ddd-96cd-b9e37ba9ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.699495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d570fc-03da-4ddd-96cd-b9e37ba9ddd1 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.700202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d570fc-03da-4ddd-96cd-b9e37ba9ddd1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:25.700508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d570fc-03da-4ddd-96cd-b9e37ba9ddd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.701886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d570fc-03da-4ddd-96cd-b9e37ba9ddd1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.712817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e11a8-526b-4542-a4fb-4e1f8519d829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.713022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 467/950] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:25.714490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e11a8-526b-4542-a4fb-4e1f8519d829 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.714863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e3b9-f114-46d7-ad89-12a4cb1a9079 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.714968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e11a8-526b-4542-a4fb-4e1f8519d829 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.715203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e3b9-f114-46d7-ad89-12a4cb1a9079 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.715385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e11a8-526b-4542-a4fb-4e1f8519d829 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.715542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e3b9-f114-46d7-ad89-12a4cb1a9079 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.719119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 468/951] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:25.719119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e3b9-f114-46d7-ad89-12a4cb1a9079 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.725768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e11a8-526b-4542-a4fb-4e1f8519d829 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.727267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e11a8-526b-4542-a4fb-4e1f8519d829 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-653e11a8-526b-4542-a4fb-4e1f8519d829 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:25.739807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:25.741277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:25.742988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:25.746727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 484/952] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:25.748186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe73dd2-21b9-4d91-99ba-bff5a14fd540 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.748603 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe73dd2-21b9-4d91-99ba-bff5a14fd540 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.748875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe73dd2-21b9-4d91-99ba-bff5a14fd540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.756339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:25.771125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-dbe73dd2-21b9-4d91-99ba-bff5a14fd540 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:25.772651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dbe73dd2-21b9-4d91-99ba-bff5a14fd540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:25.827551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:25.856866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e3b9-f114-46d7-ad89-12a4cb1a9079 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:25.875150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 485/953] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:25.875150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe73dd2-21b9-4d91-99ba-bff5a14fd540 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:25.902438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:25.903374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:25.903668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:25.903941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:25.904764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:25.905070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:25.907090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:25.922094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 469/954] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:25.923210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:26.003591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe73dd2-21b9-4d91-99ba-bff5a14fd540 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:26.054081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:26.055354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.055906 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.056859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:26.057149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:26.059508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:26.060428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:26.064571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:26.067847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f982707-63c7-4a45-9684-a7d4aa3b6d99 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:26.084890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 486/955] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:26.092162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:26.110989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:26.111658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.111932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.112204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:26.113352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:26.114734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:26.147276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 470/956] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:26.172380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:26.183041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:26.248664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5027b-2f57-4844-85b1-3df1f7b74559 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:26.316187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:26.317590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.318373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.318991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/4ee3b5df27184223a916ceccb0ee4717/roles/6752b97865d54ef3a53d54dbadc24ae4` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:26.322908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=4ee3b5df27184223a916ceccb0ee4717, role_id=6752b97865d54ef3a53d54dbadc24ae4)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:26.326401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:26.343095 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74aae9-1c6a-403f-b25d-b9cf0fa921dc None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:26.353104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 487/957] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:55:26 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/4ee3b5df27184223a916ceccb0ee4717/roles/6752b97865d54ef3a53d54dbadc24ae4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:55:26.387507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:26.389749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.389749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.389749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] PATH_INFO: `/v3/services/compute` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:26.389749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:26.391052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:26.400897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:26.402300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:26.403935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 471/958] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:26.409320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:26.533340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:26.545965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c551a-472d-4be2-abf8-100858a59084 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:26.597669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1bb1-51ac-474f-96b2-4845906c44b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:26.598213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1bb1-51ac-474f-96b2-4845906c44b9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.598493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1bb1-51ac-474f-96b2-4845906c44b9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.598774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1bb1-51ac-474f-96b2-4845906c44b9 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:26.599590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded1bb1-51ac-474f-96b2-4845906c44b9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:26.600073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded1bb1-51ac-474f-96b2-4845906c44b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:26.602362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded1bb1-51ac-474f-96b2-4845906c44b9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:26.611660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 488/959] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:26.618328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:26.638256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:26.740441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c091464-0a48-4a49-bcb6-caf9da7cae47 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:26.779813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018785c-ab1f-4f23-88f3-cce264fc48e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:26.780435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018785c-ab1f-4f23-88f3-cce264fc48e5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.780714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018785c-ab1f-4f23-88f3-cce264fc48e5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.780984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018785c-ab1f-4f23-88f3-cce264fc48e5 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:26.781628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018785c-ab1f-4f23-88f3-cce264fc48e5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:26.781933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018785c-ab1f-4f23-88f3-cce264fc48e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:26.783460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018785c-ab1f-4f23-88f3-cce264fc48e5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:26.789839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 472/960] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:26.983625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe912b4-bee9-4d24-b389-d16cbbfb11c0 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.984289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe912b4-bee9-4d24-b389-d16cbbfb11c0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.984780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe912b4-bee9-4d24-b389-d16cbbfb11c0 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:26.986644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 489/961] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:26.993008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d40e10-45c5-49e3-b785-507589f81f06 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:26.993380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d40e10-45c5-49e3-b785-507589f81f06 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:26.993626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d40e10-45c5-49e3-b785-507589f81f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:27.014772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-05d40e10-45c5-49e3-b785-507589f81f06 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:27.016090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-05d40e10-45c5-49e3-b785-507589f81f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:27.103213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 473/962] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:27.438549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe912b4-bee9-4d24-b389-d16cbbfb11c0 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:27.558339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe912b4-bee9-4d24-b389-d16cbbfb11c0 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:27.602046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:27.603458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:27.604212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:27.604922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:27.606180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:27.608775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:27.630677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.sql.core [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:55:27.630677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 18:55:27.630677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [parameters: {'user_id': 'de92f56ecf794afeb0b5971e360bdda1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 18:55:27.630677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=76733) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result.read() Aug 29 18:55:27.640339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:55:27.641299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application yield resource Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application next(self.gen) Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise self.value Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application session.commit() Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 18:55:27.642260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.session.flush() Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application compat.raise_( Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 18:55:27.643223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 18:55:27.644694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application conn.query(q) Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application result.read() Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 18:55:27.647095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [parameters: {'user_id': 'de92f56ecf794afeb0b5971e360bdda1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:27.649418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 18:55:27.651527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 18:55:27.653917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 18:55:27.653917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 18:55:27.653917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 18:55:27.653917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 18:55:27.653917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 18:55:27.653917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:27.653917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 490/963] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Aug 29 18:55:27 2022] POST /identity/v3/users => generated 160 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:27.654961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d40e10-45c5-49e3-b785-507589f81f06 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:27.666221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:27.779161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d40e10-45c5-49e3-b785-507589f81f06 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:27.829597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dc76b-80eb-4734-a0b1-b15d64576a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:27.830573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dc76b-80eb-4734-a0b1-b15d64576a77 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:27.835146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dc76b-80eb-4734-a0b1-b15d64576a77 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:27.835146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dc76b-80eb-4734-a0b1-b15d64576a77 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:27.847291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dc76b-80eb-4734-a0b1-b15d64576a77 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:27.847291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dc76b-80eb-4734-a0b1-b15d64576a77 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-840dc76b-80eb-4734-a0b1-b15d64576a77 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:27.858184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:27.859217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 18:55:27.860209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:27.864439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 474/964] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:55:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:27.866432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:27.894294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:27.995858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59920783-489d-433e-9090-fde109ceb3f0 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:28.038036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805aabaf-f754-4371-900d-dfd3b16c8e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:28.038987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aabaf-f754-4371-900d-dfd3b16c8e12 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:28.039504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aabaf-f754-4371-900d-dfd3b16c8e12 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:28.040001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aabaf-f754-4371-900d-dfd3b16c8e12 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:28.040913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805aabaf-f754-4371-900d-dfd3b16c8e12 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:28.041429 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805aabaf-f754-4371-900d-dfd3b16c8e12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:28.043936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805aabaf-f754-4371-900d-dfd3b16c8e12 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:28.089134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 491/965] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Aug 29 18:55:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:28.463468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784638f-3061-4779-8197-0529498b6e6c None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:28.464427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784638f-3061-4779-8197-0529498b6e6c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:28.464761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784638f-3061-4779-8197-0529498b6e6c None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:28.466698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 475/966] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:28.472388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01639660-0a77-4232-a90b-76897cae0526 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:28.472921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01639660-0a77-4232-a90b-76897cae0526 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:28.473061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01639660-0a77-4232-a90b-76897cae0526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:28.494449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-01639660-0a77-4232-a90b-76897cae0526 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:28.496038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-01639660-0a77-4232-a90b-76897cae0526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:28.564046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4d4b6-4bd2-4bf0-b054-0fb2ecca9fc7 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:28.564328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4d4b6-4bd2-4bf0-b054-0fb2ecca9fc7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:28.564602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4d4b6-4bd2-4bf0-b054-0fb2ecca9fc7 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:28.566063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 476/967] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:28.571669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72f3f-d0b5-486b-9182-56cab581d0f8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:28.571957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72f3f-d0b5-486b-9182-56cab581d0f8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:28.572237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72f3f-d0b5-486b-9182-56cab581d0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:28.595036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a1e72f3f-d0b5-486b-9182-56cab581d0f8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:28.596055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 492/968] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:28.596385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a1e72f3f-d0b5-486b-9182-56cab581d0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:28.673559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:28.681152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 477/969] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:28.879241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01639660-0a77-4232-a90b-76897cae0526 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:28.888912 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:28.960953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72f3f-d0b5-486b-9182-56cab581d0f8 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.017250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01639660-0a77-4232-a90b-76897cae0526 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.055456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6634053-a5f2-4079-8978-3692426c2de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.056094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6634053-a5f2-4079-8978-3692426c2de7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.056370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6634053-a5f2-4079-8978-3692426c2de7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.056637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6634053-a5f2-4079-8978-3692426c2de7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.057251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6634053-a5f2-4079-8978-3692426c2de7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.058889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6634053-a5f2-4079-8978-3692426c2de7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-d6634053-a5f2-4079-8978-3692426c2de7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:29.068763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:29.069963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:29.070985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:29.071925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72f3f-d0b5-486b-9182-56cab581d0f8 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.072633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 493/970] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:29.074039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99505-94f7-4b4f-bbfe-2c11e1a3d5e3 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.074323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99505-94f7-4b4f-bbfe-2c11e1a3d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.074594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99505-94f7-4b4f-bbfe-2c11e1a3d5e3 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.076091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 494/971] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:29.079059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99505-94f7-4b4f-bbfe-2c11e1a3d5e3 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.114230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6535f-3b49-433d-9bad-3dd1b0672e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.115185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6535f-3b49-433d-9bad-3dd1b0672e78 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.115633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6535f-3b49-433d-9bad-3dd1b0672e78 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.116074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6535f-3b49-433d-9bad-3dd1b0672e78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.131959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6535f-3b49-433d-9bad-3dd1b0672e78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-1ec6535f-3b49-433d-9bad-3dd1b0672e78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:29.136993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:29.137962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:29.139357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 478/972] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Aug 29 18:55:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:29.140190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963cc4-9e37-428f-9463-61e6ca035706 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.140190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963cc4-9e37-428f-9463-61e6ca035706 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.140322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963cc4-9e37-428f-9463-61e6ca035706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.157141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:29.160912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-30963cc4-9e37-428f-9463-61e6ca035706 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:29.162126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-30963cc4-9e37-428f-9463-61e6ca035706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:29.193222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99505-94f7-4b4f-bbfe-2c11e1a3d5e3 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.227864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:29.232312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.233150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.233430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.233703 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.234335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:29.234642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.236045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.245995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 495/973] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:29.245995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.246303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 479/974] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:29.247872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30963cc4-9e37-428f-9463-61e6ca035706 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.359477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8015f-cc53-4e2a-ae66-4bd9a0052476 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.385056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30963cc4-9e37-428f-9463-61e6ca035706 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.400136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.400770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.401036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.401866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.402587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:29.402943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.404376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.406070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:29.419217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 496/975] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Aug 29 18:55:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:29.424852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.432005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.432634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.432920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.433196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.433846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:29.434155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.435692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.441262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 480/976] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:29.447223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.520294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:29.563265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae771a46-7483-4547-a8cf-ca4983485a66 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.570425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4db32-2dfe-4b0d-b161-5bb603971d33 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.600077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.600717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.600995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.601264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.608352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.608975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.609255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.609530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.610249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] RBAC: Adding query filter params (interface=admin, service_id=4732d95f975045028d267e4a9a66d016, region_id=RegionOne) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:29.610563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.611975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.613534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.614976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.617905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 481/977] 10.222.0.94 () {56 vars in 1249 bytes} [Mon Aug 29 18:55:29 2022] GET /identity/v3/endpoints?service_id=4732d95f975045028d267e4a9a66d016&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:29.618735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:29.631063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:29.633021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:29.635051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:29.635051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:29.635051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:29.635051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 497/978] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:29.635051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.690718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:29.744042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14baeac4-faa6-48fc-9acd-e7ce14363a73 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.754143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c07cad-296f-44a7-b13a-e3fcd370d352 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.762744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:29.782357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.782968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.783273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.783552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.784204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.785572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.792047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.792806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.793086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.793363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.793989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.795427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:29.803794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 498/979] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:29 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:55:29.804909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:29.816284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 482/980] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:55:29 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:29.937153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973bdfe-c0ca-4765-bb3b-850bf6a04741 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:29.955155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:29.995941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:29.996577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:29.996859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:29.997136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:29.997791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:29.998095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:29.999537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:30.014219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 499/981] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:30.021122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:30.149343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa06d8c-1e9d-482f-932e-675e61eea3e3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:30.193361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:30.194394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:30.194880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:30.195401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:30.206639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:30.209516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:30.224358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:30.225350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 18:55:30.226297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:30.228685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 483/982] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:55:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:30.235136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:30.378013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8ff8e-9ed1-4849-8e11-ef3d353efbb4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:30.420600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:30.421221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:30.421502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:30.421776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:30.422516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:30.422823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:30.424295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:30.448659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 500/983] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Aug 29 18:55:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:30.450267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:30.490959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:30.592360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f83a3-56ae-4db0-aa70-92895e8664d6 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:30.630634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:30.631268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:30.631549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:30.631819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:30.715374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:30.717236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:30.770294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:30.858844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 484/984] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:30 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:30.864791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:30.903627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:30.999609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34a378-b059-4eca-9982-82d2df7f1bbb None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:31.036943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:31.037835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.038234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.038691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.051909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:31.053388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:31.075056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:31.076493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:31.077397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:31.077397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:31.077397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:31.077397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 501/985] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:31.083523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:31.205865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e851b34-28a5-4655-8374-8c56f208657a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:31.248310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:31.249183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.249469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.249747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.250413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:31.250718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:31.252128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:31.254835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:31.272881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 485/986] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:31.275770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:31.398125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:31.431587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e96308-ea49-4af6-ab5e-d9cff6fbadb5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:31.481242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:31.481883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.482155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.482426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.490123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:31.491618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:31.505325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:31.506455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:31.507382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 502/987] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:31.515619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:31.639649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14652334-a540-4098-a92c-674b88c11ed9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:31.681123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b183d-8e64-4a4f-8ff6-d3e6739842cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:31.681630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b183d-8e64-4a4f-8ff6-d3e6739842cd None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.681905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b183d-8e64-4a4f-8ff6-d3e6739842cd None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.682175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b183d-8e64-4a4f-8ff6-d3e6739842cd None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.683771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b183d-8e64-4a4f-8ff6-d3e6739842cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:31.684248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b183d-8e64-4a4f-8ff6-d3e6739842cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:31.685693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b183d-8e64-4a4f-8ff6-d3e6739842cd None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:31.700284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 486/988] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:31.706029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:31.828514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4c1de-272c-4956-bd3a-f4fac6196858 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:31.864425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ef4fc-549b-40c2-9cff-dba7917dad41 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.865178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ef4fc-549b-40c2-9cff-dba7917dad41 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.865178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ef4fc-549b-40c2-9cff-dba7917dad41 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.866519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 487/989] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:31.872299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93cdc-9e56-43be-9797-5335e3672156 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.872677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93cdc-9e56-43be-9797-5335e3672156 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.872968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93cdc-9e56-43be-9797-5335e3672156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.873660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814354cb-91f3-4ddc-a4b4-c2852ab7d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:31.874801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814354cb-91f3-4ddc-a4b4-c2852ab7d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.875350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814354cb-91f3-4ddc-a4b4-c2852ab7d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.875847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814354cb-91f3-4ddc-a4b4-c2852ab7d3c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.876522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814354cb-91f3-4ddc-a4b4-c2852ab7d3c9 None admin] RBAC: Adding query filter params (role.id=6752b97865d54ef3a53d54dbadc24ae4, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=4ee3b5df27184223a916ceccb0ee4717) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:31.876828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814354cb-91f3-4ddc-a4b4-c2852ab7d3c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:31.878493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814354cb-91f3-4ddc-a4b4-c2852ab7d3c9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:31.895134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 503/990] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:31 2022] GET /identity/v3/role_assignments?user.id=4ee3b5df27184223a916ceccb0ee4717&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6752b97865d54ef3a53d54dbadc24ae4 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:31.895134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab950749-d63b-4ef3-b75a-34a7c9c12b35 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.895134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab950749-d63b-4ef3-b75a-34a7c9c12b35 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.895134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab950749-d63b-4ef3-b75a-34a7c9c12b35 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.895710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-01c93cdc-9e56-43be-9797-5335e3672156 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:31.895852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 504/991] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:31.895964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-01c93cdc-9e56-43be-9797-5335e3672156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:31.900866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e6d93-c908-494d-aaab-b244329818fa None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:31.901645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e6d93-c908-494d-aaab-b244329818fa None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:31.901936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e6d93-c908-494d-aaab-b244329818fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:31.909978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:31.924633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-909e6d93-c908-494d-aaab-b244329818fa None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:31.925851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-909e6d93-c908-494d-aaab-b244329818fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:31.963454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:31.988297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 488/992] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:32.022261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 505/993] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:32.262377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c93cdc-9e56-43be-9797-5335e3672156 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.296379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e6d93-c908-494d-aaab-b244329818fa None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.383840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c93cdc-9e56-43be-9797-5335e3672156 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:32.421192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e6d93-c908-494d-aaab-b244329818fa None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:32.423177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:32.423831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:32.424108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:32.424377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:32.424969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:32.426338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:32.440279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:32.441561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:32.442399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 489/994] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Aug 29 18:55:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:32.443848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.468309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:32.476484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:32.477745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:32.478306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:32.478899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] PATH_INFO: `/v3/services/compute` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:32.480527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:32.482974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:32.493306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:32.494591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:32.496256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:32.497213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 506/995] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:32.502580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.505557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:32.571330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d408b5-02cc-4b44-a699-e1803bf6a7c2 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:32.620193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:32.621071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:32.621376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:32.621652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:32.622296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:32.622635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:32.624916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:32.636365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 490/996] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:32.642869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.647267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53da6c-b8b3-4195-a051-541ae9435910 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:32.690166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:32.690786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:32.691100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:32.691363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:32.691995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] RBAC: Adding query filter params (name=compute) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:32.692299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:32.693909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:32.704261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 507/997] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:32.705348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.771103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fe863-0a0c-4b92-808a-5cbb1e2e4ca2 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:32.773323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:32.790626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:32.821181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:32.821812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:32.822042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:32.822320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:32.823785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:32.824268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:32.825288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb828c76-1f22-4617-be5d-6aaeb3caac5a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:32.825673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:32.831947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 491/998] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Aug 29 18:55:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:32.843512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.888891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:32.888891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:32.888891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:32.889605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:32.890369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:32.890879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:32.893332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:32.901640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 508/999] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:32.909976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:32.981146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109467-81d4-48f3-ba60-2b2ee9c57a76 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:33.021371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:33.021998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:33.022313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:33.022545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:33.023448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:33.024646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:33.047376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba05d-bde1-4c29-bd60-a582f92a0fe6 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:33.049433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 492/1000] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:33.079762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:33.105749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d6e7a-0da0-44e6-a5a9-6d6318c0ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:33.106427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d6e7a-0da0-44e6-a5a9-6d6318c0ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:33.107222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d6e7a-0da0-44e6-a5a9-6d6318c0ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:33.107222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d6e7a-0da0-44e6-a5a9-6d6318c0ffb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:33.107954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d6e7a-0da0-44e6-a5a9-6d6318c0ffb1 None admin] RBAC: Adding query filter params (interface=public, service_id=0ad8656fa6bb4e5dbf7ec16dceb9a08e, region_id=RegionOne) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:33.108370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d6e7a-0da0-44e6-a5a9-6d6318c0ffb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:33.109887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d6e7a-0da0-44e6-a5a9-6d6318c0ffb1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:33.116604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 509/1001] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:55:32 2022] GET /identity/v3/endpoints?service_id=0ad8656fa6bb4e5dbf7ec16dceb9a08e&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:33.122648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:33.186078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:33.269971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160b675-4952-4e8b-a5a9-2f08df1b2a44 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:33.309611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d6038-5def-4a81-b8d5-9e9468ce02b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:33.310416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d6038-5def-4a81-b8d5-9e9468ce02b9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:33.310795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d6038-5def-4a81-b8d5-9e9468ce02b9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:33.311204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d6038-5def-4a81-b8d5-9e9468ce02b9 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:33.311949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d6038-5def-4a81-b8d5-9e9468ce02b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:33.313420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d6038-5def-4a81-b8d5-9e9468ce02b9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:33.323164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 493/1002] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:33 2022] GET /identity/v3/services => generated 1727 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:55:33.927846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c2bd2-5acb-4ccd-959a-b60af403a0d3 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:33.928364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c2bd2-5acb-4ccd-959a-b60af403a0d3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:33.928517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c2bd2-5acb-4ccd-959a-b60af403a0d3 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:33.929748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 510/1003] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:33.936088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4b6ab-ba5d-432a-866c-4ca980d40a92 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:33.936369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4b6ab-ba5d-432a-866c-4ca980d40a92 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:33.936654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4b6ab-ba5d-432a-866c-4ca980d40a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:33.955876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-46b4b6ab-ba5d-432a-866c-4ca980d40a92 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:33.957105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-46b4b6ab-ba5d-432a-866c-4ca980d40a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:34.046230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 494/1004] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:34.314654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c2bd2-5acb-4ccd-959a-b60af403a0d3 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:34.427888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c2bd2-5acb-4ccd-959a-b60af403a0d3 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:34.471235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:34.471876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:34.472155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:34.472428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:34.473015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:34.474380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:34.484465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:34.485620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:34.487167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 511/1005] 10.222.0.94 () {56 vars in 1105 bytes} [Mon Aug 29 18:55:34 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:34.491817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4b6ab-ba5d-432a-866c-4ca980d40a92 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:34.523262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:34.609877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4b6ab-ba5d-432a-866c-4ca980d40a92 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:34.663905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01e668-8e23-4f86-973d-f90eadac3a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:34.664533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01e668-8e23-4f86-973d-f90eadac3a05 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:34.664812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01e668-8e23-4f86-973d-f90eadac3a05 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:34.665087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01e668-8e23-4f86-973d-f90eadac3a05 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:34.665739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01e668-8e23-4f86-973d-f90eadac3a05 None admin] RBAC: Adding query filter params (name=image) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:34.666043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01e668-8e23-4f86-973d-f90eadac3a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:34.668382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01e668-8e23-4f86-973d-f90eadac3a05 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:34.674655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 495/1006] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:34.680295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:34.742567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:34.810147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a96859-519c-41d6-9bce-3902281e92e9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:34.858435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627e9a2-96dd-42f2-b12d-87f3ca12b5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:34.859477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627e9a2-96dd-42f2-b12d-87f3ca12b5e9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:34.860290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627e9a2-96dd-42f2-b12d-87f3ca12b5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:34.860457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627e9a2-96dd-42f2-b12d-87f3ca12b5e9 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:34.861347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627e9a2-96dd-42f2-b12d-87f3ca12b5e9 None admin] RBAC: Adding query filter params (type=image) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:34.861801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627e9a2-96dd-42f2-b12d-87f3ca12b5e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:34.863358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627e9a2-96dd-42f2-b12d-87f3ca12b5e9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:34.875127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 512/1007] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:35.129928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e2bb6-083a-4a96-b71f-e0bd7e06287e None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:35.130449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e2bb6-083a-4a96-b71f-e0bd7e06287e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:35.130449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e2bb6-083a-4a96-b71f-e0bd7e06287e None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:35.132065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 496/1008] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:35.137892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665378fb-c3ef-477f-b2f7-bbee514be5a6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:35.138865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665378fb-c3ef-477f-b2f7-bbee514be5a6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:35.138865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665378fb-c3ef-477f-b2f7-bbee514be5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:35.159144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-665378fb-c3ef-477f-b2f7-bbee514be5a6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:35.160348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-665378fb-c3ef-477f-b2f7-bbee514be5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:35.246084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 513/1009] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:35.317545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969884e-f249-4fe7-b8e9-fe588af1b3fb None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:35.317929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969884e-f249-4fe7-b8e9-fe588af1b3fb None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:35.318109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969884e-f249-4fe7-b8e9-fe588af1b3fb None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:35.319678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 497/1010] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:35.325540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e0b15-e837-4746-b5e1-8c4b1fc0434c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:35.325831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e0b15-e837-4746-b5e1-8c4b1fc0434c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:35.326105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e0b15-e837-4746-b5e1-8c4b1fc0434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:35.346613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cf3e0b15-e837-4746-b5e1-8c4b1fc0434c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:35.347863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cf3e0b15-e837-4746-b5e1-8c4b1fc0434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:35.397063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:35.434135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 514/1011] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:35.519940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969884e-f249-4fe7-b8e9-fe588af1b3fb None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:35.643106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:35.650984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969884e-f249-4fe7-b8e9-fe588af1b3fb None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:35.700316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:35.701122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:35.701402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:35.701694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] PATH_INFO: `/v3/domains/default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:35.702836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:35.704319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:35.706107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 498/1012] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:35.711698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e0b15-e837-4746-b5e1-8c4b1fc0434c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:35.720335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:35.843818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e0b15-e837-4746-b5e1-8c4b1fc0434c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:35.858108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c193a91-e604-46f3-be29-8d20dd1eb8af None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:35.881950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:35.882891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:35.883380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:35.883870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:35.884734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:35.887169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:35.907213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:35.907678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:35.907941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:35.908260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:35.908889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:35.910872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:35.918860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 515/1013] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:55:35 2022] POST /identity/v3/projects => generated 311 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:35.922593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:35.923772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:35.928292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 499/1014] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:35.930242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:35.964313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:36.072358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7fe0-2c3f-496b-929d-e6e99e2f2a5d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:36.109990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:36.110616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:36.110896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:36.111206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:36.111848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] RBAC: Adding query filter params (name=compute) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:36.112170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:36.113537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:36.118879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 516/1015] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:36.125199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:36.160722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:36.258997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700716fd-5248-4576-810d-737fa5488ba2 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:36.297639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cb8f2-8db7-4d60-8386-4e6d74929ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:36.298344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cb8f2-8db7-4d60-8386-4e6d74929ee9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:36.298653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cb8f2-8db7-4d60-8386-4e6d74929ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:36.298964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cb8f2-8db7-4d60-8386-4e6d74929ee9 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:36.299695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cb8f2-8db7-4d60-8386-4e6d74929ee9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:36.299999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cb8f2-8db7-4d60-8386-4e6d74929ee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:36.301492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cb8f2-8db7-4d60-8386-4e6d74929ee9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:36.311845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 500/1016] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Aug 29 18:55:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:36.313672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:36.425595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:36.441235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5691-56dc-42df-bb5f-ed22b7451155 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:36.487995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154369a-665a-4a5a-bb7a-5006cba05e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:36.488928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154369a-665a-4a5a-bb7a-5006cba05e30 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:36.489409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154369a-665a-4a5a-bb7a-5006cba05e30 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:36.489859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154369a-665a-4a5a-bb7a-5006cba05e30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:36.490971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154369a-665a-4a5a-bb7a-5006cba05e30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:36.493375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154369a-665a-4a5a-bb7a-5006cba05e30 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:36.528970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 517/1017] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 18:55:36.893895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4586cb-5830-47c3-8bc5-e2bf6eb0b416 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:36.894224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4586cb-5830-47c3-8bc5-e2bf6eb0b416 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:36.894454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4586cb-5830-47c3-8bc5-e2bf6eb0b416 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:36.896155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 501/1018] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:36.901924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037ee5-5220-4767-a585-8bdb2ef78af2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:36.902216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037ee5-5220-4767-a585-8bdb2ef78af2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:36.902482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037ee5-5220-4767-a585-8bdb2ef78af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:36.922821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d2037ee5-5220-4767-a585-8bdb2ef78af2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:36.924076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d2037ee5-5220-4767-a585-8bdb2ef78af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:37.019826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 518/1019] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:36 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:37.296030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4586cb-5830-47c3-8bc5-e2bf6eb0b416 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:37.396782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:37.408774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4586cb-5830-47c3-8bc5-e2bf6eb0b416 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:37.444829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8cf9a-3e8d-4cf5-93af-56866cf7aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:37.445444 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8cf9a-3e8d-4cf5-93af-56866cf7aae6 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:37.445722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8cf9a-3e8d-4cf5-93af-56866cf7aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:37.445991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8cf9a-3e8d-4cf5-93af-56866cf7aae6 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:37.446657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8cf9a-3e8d-4cf5-93af-56866cf7aae6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:37.448191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8cf9a-3e8d-4cf5-93af-56866cf7aae6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:37.466839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 502/1020] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:55:37 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:38.034609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ea6ca-eeb2-4fe6-9c18-f29ab3070081 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:38.035438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ea6ca-eeb2-4fe6-9c18-f29ab3070081 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:38.035438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ea6ca-eeb2-4fe6-9c18-f29ab3070081 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:38.036743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 519/1021] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:38.042996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c683e-963b-4877-939a-4fd64c6964fd None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:38.043313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c683e-963b-4877-939a-4fd64c6964fd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:38.043594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c683e-963b-4877-939a-4fd64c6964fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:38.064850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bf2c683e-963b-4877-939a-4fd64c6964fd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:38.066775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bf2c683e-963b-4877-939a-4fd64c6964fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:38.164139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 503/1022] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:38.442145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ea6ca-eeb2-4fe6-9c18-f29ab3070081 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:38.523850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:38.561709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ea6ca-eeb2-4fe6-9c18-f29ab3070081 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:38.601709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeb174-b791-4a7a-a1d4-c9b9759fe9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:38.602348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb174-b791-4a7a-a1d4-c9b9759fe9bf None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:38.602621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb174-b791-4a7a-a1d4-c9b9759fe9bf None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:38.602889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb174-b791-4a7a-a1d4-c9b9759fe9bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:38.604042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abeb174-b791-4a7a-a1d4-c9b9759fe9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:38.606859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abeb174-b791-4a7a-a1d4-c9b9759fe9bf None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:38.611116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 520/1023] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:38.614293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c683e-963b-4877-939a-4fd64c6964fd None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:38.656945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17035745-1dfa-4d6c-aca0-450d9d02abc2 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:38.657215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17035745-1dfa-4d6c-aca0-450d9d02abc2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:38.657488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17035745-1dfa-4d6c-aca0-450d9d02abc2 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:38.659207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 521/1024] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:38.664873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eaff6-1290-4edc-826c-1c0660bcc0ff None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:38.665159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eaff6-1290-4edc-826c-1c0660bcc0ff None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:38.665454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eaff6-1290-4edc-826c-1c0660bcc0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:38.689525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c36eaff6-1290-4edc-826c-1c0660bcc0ff None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:38.690920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c36eaff6-1290-4edc-826c-1c0660bcc0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:38.727608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:38.756485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c683e-963b-4877-939a-4fd64c6964fd None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:38.791262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 522/1025] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:38.798301 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:38.798985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:38.799294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:38.802383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:38.802383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:38.802383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:38.830635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 504/1026] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:55:38 2022] POST /identity/v3/users => generated 285 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:39.091817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eaff6-1290-4edc-826c-1c0660bcc0ff None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:39.174274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:39.216598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eaff6-1290-4edc-826c-1c0660bcc0ff None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:39.262167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:39.262838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.263190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.263769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.281115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:39.286787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:39.287827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:39.288702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 523/1027] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Aug 29 18:55:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:39.296195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:39.414812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf78891-fd0b-4c60-bd3b-6796aace3e57 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:39.453564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c3b6-4e87-4870-973f-5c801196d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:39.454207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c3b6-4e87-4870-973f-5c801196d2df None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.454508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c3b6-4e87-4870-973f-5c801196d2df None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.454806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c3b6-4e87-4870-973f-5c801196d2df None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.455570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c3b6-4e87-4870-973f-5c801196d2df None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:39.455876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c3b6-4e87-4870-973f-5c801196d2df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:39.457334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c3b6-4e87-4870-973f-5c801196d2df None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:39.458982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:39.476544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 505/1028] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Aug 29 18:55:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:39.477913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:39.554893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245d89f-9244-4bb8-85e5-a219db63a343 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.555313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245d89f-9244-4bb8-85e5-a219db63a343 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.555448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245d89f-9244-4bb8-85e5-a219db63a343 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.556951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 506/1029] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:39.562590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5186c5-67d5-4b95-a93f-3d09d93ea48d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.562888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5186c5-67d5-4b95-a93f-3d09d93ea48d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.563168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5186c5-67d5-4b95-a93f-3d09d93ea48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.582218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8a5186c5-67d5-4b95-a93f-3d09d93ea48d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:39.583457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8a5186c5-67d5-4b95-a93f-3d09d93ea48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:39.602870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc6035-97cb-4b1c-9e0a-918cecd393fc None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:39.638068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:39.648889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:39.649526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.649806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.650079 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.660175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 507/1030] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:39.661872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:39.663371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:39.684241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:39.685925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:39.687656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:39.687656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:39.687656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:39.690438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 524/1031] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:39.691303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5186c5-67d5-4b95-a93f-3d09d93ea48d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:39.815836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5186c5-67d5-4b95-a93f-3d09d93ea48d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:39.848027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:39.858451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:39.859294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:39.859573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:39.859852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:39.860543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:39.860851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:39.862290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:39.879576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 508/1032] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:39.881088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:39.949387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.040736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc69bd-559e-40b9-a17f-3c665b237102 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.090118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae5e89-a007-4e64-8cb1-f2c0d130346a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.092267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.093767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.094051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.094329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.102669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.104138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:40.122124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:40.123608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:40.124727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 525/1033] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:55:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:40.129823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.132102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.133077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.133363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.134553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] PATH_INFO: `/v3/services/image` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.135207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.136659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:40.158274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:40.159627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:40.160486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:40.161226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 509/1034] 10.222.0.94 () {56 vars in 1105 bytes} [Mon Aug 29 18:55:39 2022] GET /identity/v3/services/image => generated 86 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:40.166572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.189893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:40.234638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:40.263467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41defd5f-01e1-4b2c-86aa-33f66c8f9d1d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.300280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a8e0a-95fb-4851-9d96-4b8251ebedc5 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.304803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.305428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.305710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.305984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.306723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:40.307029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.309701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.334728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 526/1035] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:40.340675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.352713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.353648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.353929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.354379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.355033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] RBAC: Adding query filter params (name=image) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:40.355664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.357080 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.367783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 510/1036] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:40.369185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.460368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:40.466274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a2d5-1020-4b32-beca-920fbbd39436 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.489136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a67a1-abfa-48d9-9154-2c2671c884d3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.506562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:40.508035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.509131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.509491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.509915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.534995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.535747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.536034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.536530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.537250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] RBAC: Adding query filter params (type=image) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:40.537562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.538974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.546297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 511/1037] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:40.552136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.600544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.602043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.672605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e77fc-7f87-49e1-a06b-bde89e6893bb None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.719856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.720441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:40.720555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.720747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.721024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.721750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] RBAC: Adding query filter params (interface=public, service_id=00009d977ee642e590dab05d37eae52c, region_id=RegionOne) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:40.722154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.723754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.730284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 512/1038] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/endpoints?service_id=00009d977ee642e590dab05d37eae52c&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:40.733286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 527/1039] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:40.737461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.738890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.795760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:40.877887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c838-1079-4187-919b-5cb4e7b5e0c1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.879787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00474df-86a7-4eab-9cbd-df0cca86b78e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:40.920651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.930122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.930122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.930122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.936001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.937562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.939314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f9a42-0a47-49fd-a620-507d05cbe829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:40.939937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f9a42-0a47-49fd-a620-507d05cbe829 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.940217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f9a42-0a47-49fd-a620-507d05cbe829 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.940486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f9a42-0a47-49fd-a620-507d05cbe829 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.941106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f9a42-0a47-49fd-a620-507d05cbe829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:40.942472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f9a42-0a47-49fd-a620-507d05cbe829 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:40.950008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 513/1040] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/services => generated 1957 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 18:55:40.951259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c7352-1c4a-455e-8838-e0b09ff2a944 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.951546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c7352-1c4a-455e-8838-e0b09ff2a944 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.951817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c7352-1c4a-455e-8838-e0b09ff2a944 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.955121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 514/1041] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:40.956233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:40.957597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:40.958957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:40.958957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:40.958957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:40.959465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 528/1042] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:40.961039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ede3d6-4240-4fb1-8dd5-c6b25f4de78b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:40.961325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ede3d6-4240-4fb1-8dd5-c6b25f4de78b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:40.961611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ede3d6-4240-4fb1-8dd5-c6b25f4de78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:40.963779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:40.982938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e4ede3d6-4240-4fb1-8dd5-c6b25f4de78b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:40.984582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e4ede3d6-4240-4fb1-8dd5-c6b25f4de78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:41.030761 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:41.090574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 515/1043] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:41.091339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8a76a-20d5-4ecb-8414-71299d30dabd None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:41.134036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:41.134658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.134918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.135256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.136067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:41.136343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:41.137744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:41.152375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 529/1044] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:41.160326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ede3d6-4240-4fb1-8dd5-c6b25f4de78b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:41.199325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:41.277768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ede3d6-4240-4fb1-8dd5-c6b25f4de78b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:41.323137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:41.323938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.324208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.324446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.337693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:41.340403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:41.356190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:41.357172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:41.358311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 516/1045] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:41.365472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:41.367290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:41.516316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:41.523142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fee5a-8b3b-49f8-8ee7-ca0e1184df47 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:41.528731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e73ace-ae0f-49c0-9c2c-82c907580bd0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:41.540752 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:41.578113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:41.579512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.580125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.580661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.581804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:41.582288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:41.583184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:41.584147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.584497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.584837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.585041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:41.597870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:41.602928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:41.604101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:41.610832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 517/1046] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:41.613994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:41.614693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 530/1047] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:41.615423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:41.748796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d318fd-2e46-4e7b-b4d8-cd9231fa8bfa None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:41.765509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7755-521d-44b2-b167-7733222beeda None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:41.806456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:41.807126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.808094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.808094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.809035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:41.809352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:41.811402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:41.815205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:41.816133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:41.816609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:41.817094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:41.818609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] RBAC: Adding query filter params (role.id=6752b97865d54ef3a53d54dbadc24ae4, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=8adead3860f144099efa11cefa83a37f) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:41.819427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:41.827732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 518/1048] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:55:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:41.828007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:41.829272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:41.839656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 531/1049] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:41 2022] GET /identity/v3/role_assignments?user.id=8adead3860f144099efa11cefa83a37f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6752b97865d54ef3a53d54dbadc24ae4 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:41.888291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:41.996691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3fd75-bc3b-414b-b73a-f326e4ec4727 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:42.039599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:42.040249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:42.040534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:42.040811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] PATH_INFO: `/v3/users/3267d5e4940d40ed9260f81a26e4e04a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:42.066017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] RBAC: Authorizing `identity:get_user(user_id=3267d5e4940d40ed9260f81a26e4e04a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:42.067535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:42.069298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 519/1050] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:55:41 2022] GET /identity/v3/users/3267d5e4940d40ed9260f81a26e4e04a => generated 285 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:42.076302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:42.118232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:42.238022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7477d-57d9-4ff9-856c-3983e889a8fd None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:42.303943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:42.304562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:42.304842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:42.305116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:42.379689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:42.381177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:42.421662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:42.511450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 532/1051] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:42 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:42.513385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:42.611290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:42.637744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920aa27d-6653-442a-b057-7c344515e31d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:42.676550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a19520-e371-4043-aa68-3fad01c56b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:42.677184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a19520-e371-4043-aa68-3fad01c56b96 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:42.677450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a19520-e371-4043-aa68-3fad01c56b96 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:42.677726 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a19520-e371-4043-aa68-3fad01c56b96 None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:42.678945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a19520-e371-4043-aa68-3fad01c56b96 None admin] RBAC: Authorizing `identity:get_project(project_id=3571e750788e4e5bb66f762923bb5c36)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:42.680419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a19520-e371-4043-aa68-3fad01c56b96 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:42.682234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 520/1052] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:55:42 2022] GET /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:42.688944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:42.817301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c449e-82fc-4c71-b16c-462a251ca2e8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:42.866668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0983d11-a629-4026-b7dd-d1d164410c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:42.867710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0983d11-a629-4026-b7dd-d1d164410c68 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:42.868051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0983d11-a629-4026-b7dd-d1d164410c68 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:42.868527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0983d11-a629-4026-b7dd-d1d164410c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:42.869709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0983d11-a629-4026-b7dd-d1d164410c68 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=3571e750788e4e5bb66f762923bb5c36, user.id=3267d5e4940d40ed9260f81a26e4e04a) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:42.870240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0983d11-a629-4026-b7dd-d1d164410c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:42.872726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0983d11-a629-4026-b7dd-d1d164410c68 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:42.875341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:42.887418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 533/1053] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:42 2022] GET /identity/v3/role_assignments?user.id=3267d5e4940d40ed9260f81a26e4e04a&scope.project.id=3571e750788e4e5bb66f762923bb5c36&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:43.034155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b697af2-6da4-4a5c-8f01-5977b82f57e5 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:43.034665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b697af2-6da4-4a5c-8f01-5977b82f57e5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:43.034767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b697af2-6da4-4a5c-8f01-5977b82f57e5 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:43.036262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 521/1054] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:43.041671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505304a-09c7-45d4-bb99-6bddf75996c9 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:43.042040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505304a-09c7-45d4-bb99-6bddf75996c9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:43.042319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505304a-09c7-45d4-bb99-6bddf75996c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:43.062137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1505304a-09c7-45d4-bb99-6bddf75996c9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:43.063515 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1505304a-09c7-45d4-bb99-6bddf75996c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:43.143574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 534/1055] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:43.429187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b697af2-6da4-4a5c-8f01-5977b82f57e5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:43.563245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b697af2-6da4-4a5c-8f01-5977b82f57e5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:43.611491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:43.612506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:43.612796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:43.613071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] PATH_INFO: `/v3/services/image` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:43.613679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:43.615060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:43.625208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:43.626280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 18:55:43.628191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:43.631552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 522/1056] 10.222.0.94 () {56 vars in 1105 bytes} [Mon Aug 29 18:55:43 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:43.632551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505304a-09c7-45d4-bb99-6bddf75996c9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:43.741667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505304a-09c7-45d4-bb99-6bddf75996c9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:43.772988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:43.783654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ad217-fedc-483c-b285-957ba56fbc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:43.784287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ad217-fedc-483c-b285-957ba56fbc53 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:43.784571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ad217-fedc-483c-b285-957ba56fbc53 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:43.784843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ad217-fedc-483c-b285-957ba56fbc53 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:43.785489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ad217-fedc-483c-b285-957ba56fbc53 None admin] RBAC: Adding query filter params (name=image) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:43.785793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ad217-fedc-483c-b285-957ba56fbc53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:43.787205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ad217-fedc-483c-b285-957ba56fbc53 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:43.793402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 535/1057] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:43.802461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:43.817422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:43.898528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae4d54-39db-424c-a9a4-e870e475e0ab None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:43.898812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae4d54-39db-424c-a9a4-e870e475e0ab None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:43.899117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae4d54-39db-424c-a9a4-e870e475e0ab None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:43.900491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 536/1058] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:43.906565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac131-5054-46a0-997e-36e861b28074 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:43.906863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac131-5054-46a0-997e-36e861b28074 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:43.907010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac131-5054-46a0-997e-36e861b28074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:43.928077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-463ac131-5054-46a0-997e-36e861b28074 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:43.929674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-463ac131-5054-46a0-997e-36e861b28074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:43.956214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ede658-5c5c-4a07-b9db-9e7fadbd6fe9 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:44.004778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:44.005606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.005888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.006161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.006964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] RBAC: Adding query filter params (type=image) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:44.007692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:44.009235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:44.020695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 523/1059] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:44.020916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 537/1060] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:44.024261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:44.125590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:44.161614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97429c0-3f1e-4dba-943f-65bacc3a43c1 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:44.204028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:44.204652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.204934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.205208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.205894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:44.207291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:44.236050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 524/1061] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:55:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:44.295967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ac131-5054-46a0-997e-36e861b28074 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:44.433962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ac131-5054-46a0-997e-36e861b28074 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:44.460274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:44.486639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:44.487324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.487608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.487884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.508344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:44.514699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:44.516337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:44.517175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 538/1062] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Aug 29 18:55:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:44.522268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:44.644854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdc888-3567-4d55-b8d4-01ec0370abdc None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:44.687051 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:44.701660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445059b-3889-46e1-a643-b5ec0d2226fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:44.702651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445059b-3889-46e1-a643-b5ec0d2226fa None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.703153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445059b-3889-46e1-a643-b5ec0d2226fa None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.703609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445059b-3889-46e1-a643-b5ec0d2226fa None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.704807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445059b-3889-46e1-a643-b5ec0d2226fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:44.705194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445059b-3889-46e1-a643-b5ec0d2226fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:44.707439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445059b-3889-46e1-a643-b5ec0d2226fa None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:44.718487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 525/1063] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Aug 29 18:55:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:44.725463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:44.858053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af009581-5d02-4838-bcc9-522461985db1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:44.878428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cb6c4-fe97-459d-bd35-6fb5ca881599 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.878682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cb6c4-fe97-459d-bd35-6fb5ca881599 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.878925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cb6c4-fe97-459d-bd35-6fb5ca881599 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.880407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 526/1064] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:55:44.885654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6647375-405c-4ea1-b137-b78dffe153b2 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.886685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6647375-405c-4ea1-b137-b78dffe153b2 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.886826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6647375-405c-4ea1-b137-b78dffe153b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.909510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:44.910427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:44.910856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:44.911084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f6647375-405c-4ea1-b137-b78dffe153b2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:44.911320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:44.912506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f6647375-405c-4ea1-b137-b78dffe153b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:44.922638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:44.924197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:44.940010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:44.941030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:44.941921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:44.941921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:44.941921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:44.941921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 539/1065] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:44.947001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:44.980200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:45.007122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 527/1066] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:45.073471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93696-4e9b-45eb-bd9e-862f9094587d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.117980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.118718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.119003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.119344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.119965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:45.121061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:45.122494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:45.140474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 540/1067] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:45.145958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6647375-405c-4ea1-b137-b78dffe153b2 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.263006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6647375-405c-4ea1-b137-b78dffe153b2 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.288986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:45.290527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.307759 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.308576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.308884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.309129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] PATH_INFO: `/v3/users/nova` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.317977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:45.319485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:45.334488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:45.336669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:45.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 528/1068] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:45.345036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.429142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:45.440534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef2bb0-e438-4bf2-8d6b-12a94e206cd2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.492037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.492546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d28584-ae25-48f3-89a7-fb6af96f525a None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.493078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.493530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.494017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.508806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:45.514871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:45.519337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:45.520370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 541/1069] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:45.522032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.556207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.556821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.557164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.557430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.558292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:45.558599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:45.560228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:45.580672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 529/1070] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:45.586900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.647812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:45.655183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719288b-62f3-4f03-913a-94fd0548ac6e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.699184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.699916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.700201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.700479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.701158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:45.701616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:45.703513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:45.715425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 542/1071] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:45.719162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.739197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:45.744096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e48d0-e823-413b-b800-a375713195d5 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.804944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.805968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.806430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.806889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.825870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:45.828893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:45.861286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:45.862555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:45.863813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:45.863813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:45.863813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:45.863813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 530/1072] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:45.864736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.886515 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a2f5-fd12-469c-851b-90f079d5f490 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:45.930839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:45.931541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:45.931816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:45.935913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] PATH_INFO: `/v3/users/3267d5e4940d40ed9260f81a26e4e04a` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:45.935913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] RBAC: Authorizing `identity:get_user(user_id=3267d5e4940d40ed9260f81a26e4e04a)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:45.935913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:45.936482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 543/1073] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/users/3267d5e4940d40ed9260f81a26e4e04a => generated 285 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:45.942344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:45.991763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:46.004719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12ea46-6f9d-4052-8b5f-9fa26ccc842d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:46.045987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:46.046607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.046894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.047202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.048286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:46.048562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:46.050195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:46.053380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:46.065842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 531/1074] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:46.071489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:46.082130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a29933-80fc-40cd-8c22-fc6229b71222 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:46.124464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:46.125137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.125422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.125704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.126874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] RBAC: Authorizing `identity:get_project(project_id=3571e750788e4e5bb66f762923bb5c36)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:46.128449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:46.130246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 544/1075] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:55:45 2022] GET /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:46.136263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:46.210889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0386e-f35a-41ff-a7b9-314f841ad831 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:46.256734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:46.257375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.257661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.257919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.262693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc8aa-ab4e-44a8-8412-b8b3a4a34611 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:46.266318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:46.267807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:46.280369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:46.281396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:46.282474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 532/1076] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:46.287249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:46.303733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:46.315359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899ebb5-4761-480f-896a-5a53194793a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:46.316539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ebb5-4761-480f-896a-5a53194793a8 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.316908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ebb5-4761-480f-896a-5a53194793a8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.317356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ebb5-4761-480f-896a-5a53194793a8 None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36/users/3267d5e4940d40ed9260f81a26e4e04a/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.320340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899ebb5-4761-480f-896a-5a53194793a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3571e750788e4e5bb66f762923bb5c36, user_id=3267d5e4940d40ed9260f81a26e4e04a, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:46.322712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899ebb5-4761-480f-896a-5a53194793a8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:46.342508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 545/1077] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:55:46 2022] PUT /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36/users/3267d5e4940d40ed9260f81a26e4e04a/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:55:46.344957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bf46-f045-413e-bb41-e9cf88100a74 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.345238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bf46-f045-413e-bb41-e9cf88100a74 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.345516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bf46-f045-413e-bb41-e9cf88100a74 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.346898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 546/1078] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:46.357149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53803062-20a7-4d4b-a1b9-1a7040a4b476 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.358667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53803062-20a7-4d4b-a1b9-1a7040a4b476 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.358667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53803062-20a7-4d4b-a1b9-1a7040a4b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.378044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-53803062-20a7-4d4b-a1b9-1a7040a4b476 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:46.379399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-53803062-20a7-4d4b-a1b9-1a7040a4b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:46.382223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:46.428036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84d76-ef07-4265-bee9-504b6554c023 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:46.475852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:46.476864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.477322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.477766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.479119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:46.479639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:46.491101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 547/1079] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:46.491325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:46.501818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 533/1080] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:46.508567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53803062-20a7-4d4b-a1b9-1a7040a4b476 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:46.641042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53803062-20a7-4d4b-a1b9-1a7040a4b476 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:46.690613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:46.692211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.692684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.693167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/8adead3860f144099efa11cefa83a37f/roles/6752b97865d54ef3a53d54dbadc24ae4` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:46.696032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=8adead3860f144099efa11cefa83a37f, role_id=6752b97865d54ef3a53d54dbadc24ae4)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:46.697741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:46.708533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:46.718469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 548/1081] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:55:46 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/8adead3860f144099efa11cefa83a37f/roles/6752b97865d54ef3a53d54dbadc24ae4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:55:46.769251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:46.873600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:46.923606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6237dec-a1c1-48b5-b03f-67642e6f9b82 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:46.968519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:46.969221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:46.969539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:46.969928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:47.048403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:47.049884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:47.129828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:47.183283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 534/1082] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:47.186483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:47.325781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921dfa1-a783-4fd8-9b3b-f13b6aca5623 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:47.364035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e62e27-b96a-4725-9109-e72c6e0cb3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:47.364367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62e27-b96a-4725-9109-e72c6e0cb3f4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:47.364464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62e27-b96a-4725-9109-e72c6e0cb3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:47.364932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62e27-b96a-4725-9109-e72c6e0cb3f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:47.377796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e62e27-b96a-4725-9109-e72c6e0cb3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:47.379296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e62e27-b96a-4725-9109-e72c6e0cb3f4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-65e62e27-b96a-4725-9109-e72c6e0cb3f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:47.394480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:47.395625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:47.396526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:47.396526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:47.396526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:47.400631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 549/1083] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:47.402679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:47.525477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb773d5-09b1-4a56-8978-9fa920c6578c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:47.566290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991492f-5d73-4926-bcf8-17a9ef5b66c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:47.566906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991492f-5d73-4926-bcf8-17a9ef5b66c5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:47.567220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991492f-5d73-4926-bcf8-17a9ef5b66c5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:47.567493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991492f-5d73-4926-bcf8-17a9ef5b66c5 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:47.568140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991492f-5d73-4926-bcf8-17a9ef5b66c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:47.568437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991492f-5d73-4926-bcf8-17a9ef5b66c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:47.569799 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991492f-5d73-4926-bcf8-17a9ef5b66c5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:47.588135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 535/1084] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:48.408334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5c7db-ec64-4f00-a98d-fdb733433c75 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:48.408900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5c7db-ec64-4f00-a98d-fdb733433c75 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:48.408900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5c7db-ec64-4f00-a98d-fdb733433c75 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:48.410216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 550/1085] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:48.415781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde2b7b-2fca-4eb5-80b9-623020f80a50 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:48.416249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde2b7b-2fca-4eb5-80b9-623020f80a50 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:48.416339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde2b7b-2fca-4eb5-80b9-623020f80a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:48.436519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ecde2b7b-2fca-4eb5-80b9-623020f80a50 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:48.437749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ecde2b7b-2fca-4eb5-80b9-623020f80a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:48.466608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:48.517763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 536/1086] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:48.789146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8193ff-5622-4363-9048-0687313838dc None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:48.789330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8193ff-5622-4363-9048-0687313838dc None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:48.789605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8193ff-5622-4363-9048-0687313838dc None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:48.790953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 551/1087] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:48.796115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde2b7b-2fca-4eb5-80b9-623020f80a50 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:48.797971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a08a35-751c-433b-a4be-c728f1a1be75 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:48.798766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a08a35-751c-433b-a4be-c728f1a1be75 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:48.800434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a08a35-751c-433b-a4be-c728f1a1be75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:48.823050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a4a08a35-751c-433b-a4be-c728f1a1be75 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:48.825146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a4a08a35-751c-433b-a4be-c728f1a1be75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:48.878708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:48.904101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 552/1088] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:48.916949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde2b7b-2fca-4eb5-80b9-623020f80a50 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:48.954558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:48.955239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:48.955520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:48.955792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:48.972479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:48.977337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:48.978471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:55:48.980073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:48.989114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 537/1089] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:55:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:48.989438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a08a35-751c-433b-a4be-c728f1a1be75 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.104315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a08a35-751c-433b-a4be-c728f1a1be75 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.144890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.144890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.144890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.144890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.145343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:49.145343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:49.145741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:49.155919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 553/1090] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:55:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:49.161855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.186456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:49.193945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.241769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:49.274985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f822d-2ccb-417f-9963-0ea291817616 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.306461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dfdb3-d933-4147-8a62-6275eef2ad59 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.313520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.314167 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.314458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.314739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] PATH_INFO: `/v3/users/3267d5e4940d40ed9260f81a26e4e04a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.315934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] RBAC: Authorizing `identity:get_user(user_id=3267d5e4940d40ed9260f81a26e4e04a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:49.317384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:49.319001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 538/1091] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/users/3267d5e4940d40ed9260f81a26e4e04a => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:49.325802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.353557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.354202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.354611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.354894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.366459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:55:49.379994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:49.381051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 18:55:49.381887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:49.383295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 554/1092] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:49.388275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.432510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:49.451444 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2807e5-68e6-4f27-8da9-6f488d218e98 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.505072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.505681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.505957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.506226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.508728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f181cf-8a4e-4b01-8555-a51510b590d6 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.511420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:49.553254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.553254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.553254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.553254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.553756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:49.553840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:49.555231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:49.564642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 555/1093] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:49.570284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.597499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:49.600080 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:49.683897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ea31e-c642-49f3-95bd-5f248fb1f5d3 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.715594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:49.729058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.730236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.730508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.730777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.731747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 539/1094] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:55:49.738312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.746449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:49.748997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:49.772593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:49.773797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:49.774707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:49.774707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:49.774707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:49.774707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 556/1095] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:49.781016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.807589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:49.852749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de7f4-4b2e-45f6-95e8-abcfc6bb08b3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.892550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.893314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.893620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.893875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.895177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] RBAC: Authorizing `identity:get_project(project_id=3571e750788e4e5bb66f762923bb5c36)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:49.896662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:49.898351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 540/1096] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:49.905796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:49.912457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b79e-e624-4262-8bbc-af2556ef3ef5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:49.955022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:49.955614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:49.955879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:49.956146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:49.956797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:49.957098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:49.958473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:49.973600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:49.976913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 557/1097] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:49.983652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:50.022957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b3883-9e16-427a-996a-c4d5f8916e90 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:50.064002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:50.064612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:50.064896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:50.065171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:50.065940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=3571e750788e4e5bb66f762923bb5c36, user.id=3267d5e4940d40ed9260f81a26e4e04a) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:50.066251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:50.067722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:50.068595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:50.084454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 541/1098] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/role_assignments?user.id=3267d5e4940d40ed9260f81a26e4e04a&scope.project.id=3571e750788e4e5bb66f762923bb5c36&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:50.097876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001147c0-7c60-4ed0-92e0-134c2e78e752 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:50.137880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:50.138573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:50.138863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:50.139431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:50.149257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:50.150741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:55:50.165765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:50.166937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:55:50.168875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 558/1099] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Aug 29 18:55:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:55:50.177327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:50.331903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80186b02-ca80-4c5c-bc2d-e79820bc3825 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:50.363307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:50.379729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:50.380348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:50.380626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:50.380895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:50.381641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:50.381944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:50.383394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:50.405422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 542/1100] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Aug 29 18:55:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:50.411821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:50.547536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44171e6c-b586-4067-9009-28f9f262d6a0 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:50.594415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:50.594953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:50.595265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:50.595540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:50.599342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:50.671389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:50.672906 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:50.789815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 559/1101] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:50.797641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:50.928865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a736610-0d79-42f9-861b-d3f6b223d2d8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:50.970296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:50.970948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:50.971252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:50.971526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:50.984363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:50.985820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:55:51.007714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:51.008798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:55:51.009707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:55:51.009707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:55:51.009707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:51.009707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 543/1102] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:51.016466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:51.054137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:51.135321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd45bc8-a6c8-4eb7-ba33-51fc379ddf81 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:51.181471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:51.182014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:51.182300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:51.182583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:51.183548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:51.183862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:51.185273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:51.204968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 560/1103] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:51.212515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:51.259232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:51.356295 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d55aae-97a5-4174-b84b-c4903248be46 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:51.404681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:51.405612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:51.405979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:51.406352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:51.413886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:51.415361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:55:51.431268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:55:51.432496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:55:51.433922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 544/1104] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:55:51.440494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:51.586360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:51.595490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780afdfb-e3c4-42bd-bfe4-1fe70f36eb58 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:51.645817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ccdcc-d2d1-4fcd-bf5f-f662d9e89675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:51.646485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccdcc-d2d1-4fcd-bf5f-f662d9e89675 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:51.646776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccdcc-d2d1-4fcd-bf5f-f662d9e89675 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:51.647057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccdcc-d2d1-4fcd-bf5f-f662d9e89675 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:51.647786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ccdcc-d2d1-4fcd-bf5f-f662d9e89675 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:51.648150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ccdcc-d2d1-4fcd-bf5f-f662d9e89675 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:51.649618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ccdcc-d2d1-4fcd-bf5f-f662d9e89675 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:51.664243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 561/1105] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:51.671728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:51.794692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:51.816911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb66973-f954-419f-9a4e-a509b5592d3c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:51.856387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dddd3-d0d1-46d2-a84a-b7f715bb9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:51.857005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dddd3-d0d1-46d2-a84a-b7f715bb9d3d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:51.857284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dddd3-d0d1-46d2-a84a-b7f715bb9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:51.857559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dddd3-d0d1-46d2-a84a-b7f715bb9d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:51.858296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dddd3-d0d1-46d2-a84a-b7f715bb9d3d None admin] RBAC: Adding query filter params (role.id=6752b97865d54ef3a53d54dbadc24ae4, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=8adead3860f144099efa11cefa83a37f) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:51.858606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dddd3-d0d1-46d2-a84a-b7f715bb9d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:51.860074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dddd3-d0d1-46d2-a84a-b7f715bb9d3d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:51.867342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 545/1106] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:51 2022] GET /identity/v3/role_assignments?user.id=8adead3860f144099efa11cefa83a37f&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6752b97865d54ef3a53d54dbadc24ae4 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:52.138972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f1c37-77ef-4552-93a0-af19cac52448 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:52.139608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f1c37-77ef-4552-93a0-af19cac52448 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:52.139796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f1c37-77ef-4552-93a0-af19cac52448 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:52.141167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 562/1107] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:52.146836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60976bbb-f5c2-447a-90fc-fa15434e5ffd None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:52.147232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60976bbb-f5c2-447a-90fc-fa15434e5ffd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:52.147520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60976bbb-f5c2-447a-90fc-fa15434e5ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:52.168218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-60976bbb-f5c2-447a-90fc-fa15434e5ffd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:52.170116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-60976bbb-f5c2-447a-90fc-fa15434e5ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:52.258712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 546/1108] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:55:52.544396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f1c37-77ef-4552-93a0-af19cac52448 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:52.680500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f1c37-77ef-4552-93a0-af19cac52448 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:52.719918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f08bc0-0770-4174-8b9b-5c80a084e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:52.720365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f08bc0-0770-4174-8b9b-5c80a084e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:52.720652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f08bc0-0770-4174-8b9b-5c80a084e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:52.720927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f08bc0-0770-4174-8b9b-5c80a084e1b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:52.722088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f08bc0-0770-4174-8b9b-5c80a084e1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:52.723561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f08bc0-0770-4174-8b9b-5c80a084e1b0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:52.725154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 563/1109] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:55:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:52.731239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60976bbb-f5c2-447a-90fc-fa15434e5ffd None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:52.835246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:52.860470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60976bbb-f5c2-447a-90fc-fa15434e5ffd None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:52.897373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181012a-772a-41d7-9e99-162b3396400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:52.897976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181012a-772a-41d7-9e99-162b3396400f None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:52.898252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181012a-772a-41d7-9e99-162b3396400f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:52.898521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181012a-772a-41d7-9e99-162b3396400f None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:52.899265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181012a-772a-41d7-9e99-162b3396400f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:52.900635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181012a-772a-41d7-9e99-162b3396400f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:52.924966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 547/1110] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:55:52 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:55:54.954845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c657ae-b808-4a49-bcf1-b9a7b9bf110c None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:54.955411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c657ae-b808-4a49-bcf1-b9a7b9bf110c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:54.955411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c657ae-b808-4a49-bcf1-b9a7b9bf110c None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:54.956769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 564/1111] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:54.962250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3467f-dcc0-493d-8f60-b254993aa28c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:54.962713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3467f-dcc0-493d-8f60-b254993aa28c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:54.962858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3467f-dcc0-493d-8f60-b254993aa28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:54.982176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-00e3467f-dcc0-493d-8f60-b254993aa28c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:54.983413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-00e3467f-dcc0-493d-8f60-b254993aa28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:55.054222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 548/1112] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:55.328184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c657ae-b808-4a49-bcf1-b9a7b9bf110c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:55.355911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:55.446840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c657ae-b808-4a49-bcf1-b9a7b9bf110c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:55.482439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:55.483110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:55.483382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:55.483651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] PATH_INFO: `/v3/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:55.485077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] RBAC: Authorizing `identity:get_role(role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:55.486614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:55.487980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 565/1113] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:55:55 2022] GET /identity/v3/roles/377956ded99f409684f20afaaf7052c8 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:55.493778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3467f-dcc0-493d-8f60-b254993aa28c None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:55.567119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:55.604297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3467f-dcc0-493d-8f60-b254993aa28c None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:55.641211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:55.641850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:55.642129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:55.642404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] PATH_INFO: `/v3/users/446cb9461bf542908172e046f3f4cfce` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:55.656986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] RBAC: Authorizing `identity:get_user(user_id=446cb9461bf542908172e046f3f4cfce)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:55.658445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:55.660181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 549/1114] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:55:55 2022] GET /identity/v3/users/446cb9461bf542908172e046f3f4cfce => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:55.666127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:55.771191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:55.780855 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe242625-eae8-44f9-b96b-56fb46c04a9a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:55.818674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:55.819317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:55.819601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:55.819873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:55.889937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:55.891336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:55.984759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:55.994333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 566/1115] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:55:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:55:56.000775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:56.108134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9da68-838e-40b6-8264-6c8a03f44810 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:56.137629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:56.146588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355748f2-f4b1-4d21-a751-ad2f948e9d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:56.147150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355748f2-f4b1-4d21-a751-ad2f948e9d33 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:56.147435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355748f2-f4b1-4d21-a751-ad2f948e9d33 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:56.147709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355748f2-f4b1-4d21-a751-ad2f948e9d33 None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:56.148877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355748f2-f4b1-4d21-a751-ad2f948e9d33 None admin] RBAC: Authorizing `identity:get_project(project_id=3571e750788e4e5bb66f762923bb5c36)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:56.150309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355748f2-f4b1-4d21-a751-ad2f948e9d33 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:56.152914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 550/1116] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:55:55 2022] GET /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:55:56.159335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:56.264785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb994d0-ad61-4c6a-92a1-5cb1340af5bd None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:56.301651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b2330-2190-4b44-893c-8b7b8e303aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:56.302259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b2330-2190-4b44-893c-8b7b8e303aa1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:56.302536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b2330-2190-4b44-893c-8b7b8e303aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:56.302806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b2330-2190-4b44-893c-8b7b8e303aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:56.303563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b2330-2190-4b44-893c-8b7b8e303aa1 None admin] RBAC: Adding query filter params (role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=3571e750788e4e5bb66f762923bb5c36, user.id=446cb9461bf542908172e046f3f4cfce) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:55:56.303872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b2330-2190-4b44-893c-8b7b8e303aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:56.305307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b2330-2190-4b44-893c-8b7b8e303aa1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:56.311486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 567/1117] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:55:56 2022] GET /identity/v3/role_assignments?user.id=446cb9461bf542908172e046f3f4cfce&scope.project.id=3571e750788e4e5bb66f762923bb5c36&role.id=377956ded99f409684f20afaaf7052c8 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:58.282846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc13b5-479a-4b32-b6dd-150010738d6a None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:58.283415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc13b5-479a-4b32-b6dd-150010738d6a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:58.283415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc13b5-479a-4b32-b6dd-150010738d6a None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:58.284769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 551/1118] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:55:58.290225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de04d-9956-47f3-96d7-c27195c8c772 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:58.291015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de04d-9956-47f3-96d7-c27195c8c772 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:58.291015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de04d-9956-47f3-96d7-c27195c8c772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:58.309879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-590de04d-9956-47f3-96d7-c27195c8c772 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:55:58.311121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-590de04d-9956-47f3-96d7-c27195c8c772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:55:58.368058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:58.385567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 568/1119] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:55:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:55:58.656000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc13b5-479a-4b32-b6dd-150010738d6a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:58.769835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc13b5-479a-4b32-b6dd-150010738d6a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:58.806763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:58.806983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:58.807231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:58.807508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] PATH_INFO: `/v3/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:58.809517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] RBAC: Authorizing `identity:get_role(role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:58.810910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:58.812324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 552/1120] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:55:58 2022] GET /identity/v3/roles/377956ded99f409684f20afaaf7052c8 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:58.818270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590de04d-9956-47f3-96d7-c27195c8c772 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:58.942767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590de04d-9956-47f3-96d7-c27195c8c772 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:58.979990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:58.980653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:58.980938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:58.981213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] PATH_INFO: `/v3/users/446cb9461bf542908172e046f3f4cfce` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:58.982382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] RBAC: Authorizing `identity:get_user(user_id=446cb9461bf542908172e046f3f4cfce)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:58.983866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:58.985501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 569/1121] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:55:58 2022] GET /identity/v3/users/446cb9461bf542908172e046f3f4cfce => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:58.991493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:59.058279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:59.104453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c21d5-c6ca-4873-82b0-7b68d5000539 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:59.145473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa91f76-7d4c-4233-8eb1-fe33a82a8df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:59.146404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91f76-7d4c-4233-8eb1-fe33a82a8df9 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:59.146743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91f76-7d4c-4233-8eb1-fe33a82a8df9 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:59.146743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91f76-7d4c-4233-8eb1-fe33a82a8df9 None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:59.148197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa91f76-7d4c-4233-8eb1-fe33a82a8df9 None admin] RBAC: Authorizing `identity:get_project(project_id=3571e750788e4e5bb66f762923bb5c36)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:59.149671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa91f76-7d4c-4233-8eb1-fe33a82a8df9 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:59.151396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 553/1122] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:55:58 2022] GET /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:55:59.159434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:55:59.228256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:55:59.280968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6df5e-fee6-44ee-a5a2-97d6e798e98c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:55:59.316281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7a931-5a8c-4cef-86dd-81d43e97a20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:55:59.316984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a931-5a8c-4cef-86dd-81d43e97a20f None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:55:59.317209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a931-5a8c-4cef-86dd-81d43e97a20f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:55:59.317480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a931-5a8c-4cef-86dd-81d43e97a20f None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36/users/446cb9461bf542908172e046f3f4cfce/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:55:59.319643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7a931-5a8c-4cef-86dd-81d43e97a20f None admin] RBAC: Authorizing `identity:create_grant(project_id=3571e750788e4e5bb66f762923bb5c36, user_id=446cb9461bf542908172e046f3f4cfce, role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:55:59.321078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7a931-5a8c-4cef-86dd-81d43e97a20f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:55:59.336364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 570/1123] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:55:59 2022] PUT /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36/users/446cb9461bf542908172e046f3f4cfce/roles/377956ded99f409684f20afaaf7052c8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:56:01.368858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce14c-d67e-40e1-bff9-bab9bbb60236 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:01.369433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce14c-d67e-40e1-bff9-bab9bbb60236 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:01.369433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce14c-d67e-40e1-bff9-bab9bbb60236 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:01.370784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 554/1124] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:01.376882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c434bbf-6c68-459e-9699-0d56fa7f649d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:01.377375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c434bbf-6c68-459e-9699-0d56fa7f649d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:01.377465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c434bbf-6c68-459e-9699-0d56fa7f649d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:01.397380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8c434bbf-6c68-459e-9699-0d56fa7f649d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:01.398809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8c434bbf-6c68-459e-9699-0d56fa7f649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:01.507952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 571/1125] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:01.814505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce14c-d67e-40e1-bff9-bab9bbb60236 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:01.942112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce14c-d67e-40e1-bff9-bab9bbb60236 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:01.976793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:01.983468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:01.984308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:01.984594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:01.984871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] PATH_INFO: `/v3/roles/377956ded99f409684f20afaaf7052c8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:01.986387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] RBAC: Authorizing `identity:get_role(role_id=377956ded99f409684f20afaaf7052c8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:01.987804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:01.989177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 555/1126] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:01 2022] GET /identity/v3/roles/377956ded99f409684f20afaaf7052c8 => generated 223 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:01.995178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c434bbf-6c68-459e-9699-0d56fa7f649d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:02.046460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:02.107577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c434bbf-6c68-459e-9699-0d56fa7f649d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:02.145990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:02.146624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:02.146901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:02.147202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] PATH_INFO: `/v3/users/446cb9461bf542908172e046f3f4cfce` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:02.148330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] RBAC: Authorizing `identity:get_user(user_id=446cb9461bf542908172e046f3f4cfce)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:02.149765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:02.152740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 572/1127] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:01 2022] GET /identity/v3/users/446cb9461bf542908172e046f3f4cfce => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:02.158604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:02.296396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89f30b-adbf-4229-bfe5-23202d6194ad None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:02.332331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:02.332943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:02.333217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:02.333490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:02.414825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:02.416999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:02.418779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:02.524749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 556/1128] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:56:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 18:56:02.531664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:02.645055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a7627-6057-4ed0-a2fc-e90a41cbeed9 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:02.661313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:02.689824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c1ef6-29d1-44d3-ac90-30b4fa92e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:02.690200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c1ef6-29d1-44d3-ac90-30b4fa92e419 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:02.690480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c1ef6-29d1-44d3-ac90-30b4fa92e419 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:02.690755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c1ef6-29d1-44d3-ac90-30b4fa92e419 None admin] PATH_INFO: `/v3/projects/3571e750788e4e5bb66f762923bb5c36` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:02.691980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c1ef6-29d1-44d3-ac90-30b4fa92e419 None admin] RBAC: Authorizing `identity:get_project(project_id=3571e750788e4e5bb66f762923bb5c36)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:02.693444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c1ef6-29d1-44d3-ac90-30b4fa92e419 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:02.695148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 573/1129] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:56:02 2022] GET /identity/v3/projects/3571e750788e4e5bb66f762923bb5c36 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:02.701657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:02.825946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:02.829338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521726-2b0a-477c-a11b-5d71e0169e41 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:02.869787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3dc53-11a9-40c9-874f-d84ffb19828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:02.870826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3dc53-11a9-40c9-874f-d84ffb19828f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:02.871327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3dc53-11a9-40c9-874f-d84ffb19828f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:02.871775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3dc53-11a9-40c9-874f-d84ffb19828f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:02.872993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3dc53-11a9-40c9-874f-d84ffb19828f None admin] RBAC: Adding query filter params (role.id=377956ded99f409684f20afaaf7052c8, scope.project.id=3571e750788e4e5bb66f762923bb5c36, user.id=446cb9461bf542908172e046f3f4cfce) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:02.873648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3dc53-11a9-40c9-874f-d84ffb19828f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:02.876165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3dc53-11a9-40c9-874f-d84ffb19828f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:02.885280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 557/1130] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:56:02 2022] GET /identity/v3/role_assignments?user.id=446cb9461bf542908172e046f3f4cfce&scope.project.id=3571e750788e4e5bb66f762923bb5c36&role.id=377956ded99f409684f20afaaf7052c8 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:04.975158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb434-9501-4145-8b8f-a206115c37eb None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:04.975709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb434-9501-4145-8b8f-a206115c37eb None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:04.975709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb434-9501-4145-8b8f-a206115c37eb None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:04.977063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 574/1131] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:56:04.984257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda47ad4-0f60-4d1e-85da-a3e547a55bfd None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:04.984840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda47ad4-0f60-4d1e-85da-a3e547a55bfd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:04.984998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda47ad4-0f60-4d1e-85da-a3e547a55bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:05.004026 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-eda47ad4-0f60-4d1e-85da-a3e547a55bfd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:05.005232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-eda47ad4-0f60-4d1e-85da-a3e547a55bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:05.085513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 558/1132] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:56:05.367006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cb434-9501-4145-8b8f-a206115c37eb None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:05.485493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cb434-9501-4145-8b8f-a206115c37eb None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:05.521241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513a58-f6a9-4016-a929-365caafcc19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:05.521880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513a58-f6a9-4016-a929-365caafcc19f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:05.522161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513a58-f6a9-4016-a929-365caafcc19f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:05.522432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513a58-f6a9-4016-a929-365caafcc19f None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:05.523610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15513a58-f6a9-4016-a929-365caafcc19f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:05.525089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15513a58-f6a9-4016-a929-365caafcc19f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:05.527374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 575/1133] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:56:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:05.533635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda47ad4-0f60-4d1e-85da-a3e547a55bfd None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:05.638497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:05.645911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda47ad4-0f60-4d1e-85da-a3e547a55bfd None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:05.691563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4f3e8-bacf-4880-a959-065ec2acddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:05.692330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4f3e8-bacf-4880-a959-065ec2acddd8 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:05.692616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4f3e8-bacf-4880-a959-065ec2acddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:05.692890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4f3e8-bacf-4880-a959-065ec2acddd8 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:05.693569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4f3e8-bacf-4880-a959-065ec2acddd8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:05.695009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4f3e8-bacf-4880-a959-065ec2acddd8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:05.725037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 559/1134] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:56:05 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:56:07.751054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bd3e-3f18-4085-895f-e9d75b62dab7 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:07.751645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bd3e-3f18-4085-895f-e9d75b62dab7 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:07.751645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bd3e-3f18-4085-895f-e9d75b62dab7 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:07.752994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 576/1135] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:07.758892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d06ff-182f-4bb1-8bf0-3a9d805f0856 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:07.759407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d06ff-182f-4bb1-8bf0-3a9d805f0856 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:07.759499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d06ff-182f-4bb1-8bf0-3a9d805f0856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:07.780787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-574d06ff-182f-4bb1-8bf0-3a9d805f0856 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:07.782030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-574d06ff-182f-4bb1-8bf0-3a9d805f0856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:07.866687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 560/1136] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:08.150868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bd3e-3f18-4085-895f-e9d75b62dab7 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:08.206579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:08.263398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bd3e-3f18-4085-895f-e9d75b62dab7 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:08.308608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526480ec-f42d-4640-8886-86c0351b10fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:08.309357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526480ec-f42d-4640-8886-86c0351b10fb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:08.309650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526480ec-f42d-4640-8886-86c0351b10fb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:08.309929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526480ec-f42d-4640-8886-86c0351b10fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:08.311092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526480ec-f42d-4640-8886-86c0351b10fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:08.312534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526480ec-f42d-4640-8886-86c0351b10fb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:08.314129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 577/1137] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:56:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:08.319937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d06ff-182f-4bb1-8bf0-3a9d805f0856 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:08.404154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:08.430130 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d06ff-182f-4bb1-8bf0-3a9d805f0856 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:08.471328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c26fe-f300-481b-9f56-abf80231fe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:08.471951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c26fe-f300-481b-9f56-abf80231fe9f None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:08.472251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c26fe-f300-481b-9f56-abf80231fe9f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:08.472524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c26fe-f300-481b-9f56-abf80231fe9f None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:08.473199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c26fe-f300-481b-9f56-abf80231fe9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:08.474636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c26fe-f300-481b-9f56-abf80231fe9f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:08.499128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 561/1138] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:56:08 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:56:10.521926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b5c58-7bd4-4828-b06e-5dcb5c01fa2c None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:10.522462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b5c58-7bd4-4828-b06e-5dcb5c01fa2c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:10.522462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b5c58-7bd4-4828-b06e-5dcb5c01fa2c None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:10.523849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 578/1139] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:10.529946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c7e4b-270f-416b-b16b-717a33fb3c43 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:10.530406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c7e4b-270f-416b-b16b-717a33fb3c43 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:10.530587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c7e4b-270f-416b-b16b-717a33fb3c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:10.550315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-991c7e4b-270f-416b-b16b-717a33fb3c43 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:10.551571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-991c7e4b-270f-416b-b16b-717a33fb3c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:10.622221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 562/1140] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:10.895332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b5c58-7bd4-4828-b06e-5dcb5c01fa2c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:11.008375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b5c58-7bd4-4828-b06e-5dcb5c01fa2c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:11.026196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:11.055012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:11.056053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:11.056506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:11.056971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:11.070769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:56:11.075819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:56:11.077505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:56:11.079272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 579/1141] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:56:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:56:11.083051 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c7e4b-270f-416b-b16b-717a33fb3c43 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:11.149619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:11.193395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c7e4b-270f-416b-b16b-717a33fb3c43 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:11.229143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:11.230013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:11.230308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:11.230674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:11.231648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:11.231950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:11.233490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:11.242345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 563/1142] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:56:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:11.249072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:11.356755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4ba2a-22b5-4cfc-900c-47756d9e9c1c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:11.395367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:11.396011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:11.396290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:11.396563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] PATH_INFO: `/v3/users/bf73c7e47ef0423c9a781c2b5497cefc` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:11.412366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] RBAC: Authorizing `identity:get_user(user_id=bf73c7e47ef0423c9a781c2b5497cefc)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:11.413726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:11.415437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 580/1143] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:11 2022] GET /identity/v3/users/bf73c7e47ef0423c9a781c2b5497cefc => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:11.421975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:11.527674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a5216-a7f1-4193-911e-06a27346402b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:11.553526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:11.565851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:11.566465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:11.566745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:11.567018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:11.664988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:11.667680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:11.849814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 564/1144] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:56:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:56:11.856338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:11.901152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:11.978989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01013bd6-86e7-4bab-a2cf-caf2f0cbe1fe None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:12.014781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519ba70-7d7e-412f-828e-241df12e0616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:12.015713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519ba70-7d7e-412f-828e-241df12e0616 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:12.016115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519ba70-7d7e-412f-828e-241df12e0616 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:12.016392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519ba70-7d7e-412f-828e-241df12e0616 None admin] PATH_INFO: `/v3/projects/9e3d8392624e4378ac432e9fc6975af9` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:12.017577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519ba70-7d7e-412f-828e-241df12e0616 None admin] RBAC: Authorizing `identity:get_project(project_id=9e3d8392624e4378ac432e9fc6975af9)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:12.019027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519ba70-7d7e-412f-828e-241df12e0616 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:12.020716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 581/1145] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:56:11 2022] GET /identity/v3/projects/9e3d8392624e4378ac432e9fc6975af9 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:12.027825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:12.067792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:12.160507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc3cf5-d058-4105-aed8-380d23c35d2f None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:12.196179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660de726-ee97-4317-ac2d-73e418328165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:12.196786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660de726-ee97-4317-ac2d-73e418328165 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:12.197068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660de726-ee97-4317-ac2d-73e418328165 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:12.197344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660de726-ee97-4317-ac2d-73e418328165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:12.198077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660de726-ee97-4317-ac2d-73e418328165 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=9e3d8392624e4378ac432e9fc6975af9, user.id=bf73c7e47ef0423c9a781c2b5497cefc) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:12.198386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660de726-ee97-4317-ac2d-73e418328165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:12.200498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660de726-ee97-4317-ac2d-73e418328165 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:12.206161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 565/1146] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:56:12 2022] GET /identity/v3/role_assignments?user.id=bf73c7e47ef0423c9a781c2b5497cefc&scope.project.id=9e3d8392624e4378ac432e9fc6975af9&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:14.157957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2912b-c4d1-4d0c-b8a4-c2c2ddc7ade8 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:14.158465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2912b-c4d1-4d0c-b8a4-c2c2ddc7ade8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:14.158465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2912b-c4d1-4d0c-b8a4-c2c2ddc7ade8 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:14.159813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 582/1147] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:14.166429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5de23-b94e-4db1-a246-78f3a1a54ae8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:14.166948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5de23-b94e-4db1-a246-78f3a1a54ae8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:14.167374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5de23-b94e-4db1-a246-78f3a1a54ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:14.186952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-11d5de23-b94e-4db1-a246-78f3a1a54ae8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:14.188296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-11d5de23-b94e-4db1-a246-78f3a1a54ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:14.266374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 566/1148] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:14.537249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2912b-c4d1-4d0c-b8a4-c2c2ddc7ade8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:14.651328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2912b-c4d1-4d0c-b8a4-c2c2ddc7ade8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:14.653412 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:14.691467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:14.692197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:14.692478 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:14.692783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:14.705359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:56:14.710194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:56:14.711460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:56:14.712660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 583/1149] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:56:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:56:14.718271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5de23-b94e-4db1-a246-78f3a1a54ae8 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:14.738018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:14.837135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5de23-b94e-4db1-a246-78f3a1a54ae8 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:14.873693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:14.874311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:14.874589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:14.874863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:14.875563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:14.875869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:14.877238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:14.885634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 567/1150] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:56:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:14.892073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:15.004502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e570-fee3-4649-8107-c0bc8c866fb5 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:15.054710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:15.055739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:15.056208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:15.056662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] PATH_INFO: `/v3/users/bf73c7e47ef0423c9a781c2b5497cefc` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:15.058551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] RBAC: Authorizing `identity:get_user(user_id=bf73c7e47ef0423c9a781c2b5497cefc)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:15.061207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:15.064101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 584/1151] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:14 2022] GET /identity/v3/users/bf73c7e47ef0423c9a781c2b5497cefc => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:15.073341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:15.165819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:15.185761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f6bbc-726f-43c8-81e8-fb954f9e5166 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:15.220726 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38b420-c170-4f76-b35d-e7200cf2a887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:15.221361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38b420-c170-4f76-b35d-e7200cf2a887 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:15.221634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38b420-c170-4f76-b35d-e7200cf2a887 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:15.221908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38b420-c170-4f76-b35d-e7200cf2a887 None admin] PATH_INFO: `/v3/projects/9e3d8392624e4378ac432e9fc6975af9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:15.223148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38b420-c170-4f76-b35d-e7200cf2a887 None admin] RBAC: Authorizing `identity:get_project(project_id=9e3d8392624e4378ac432e9fc6975af9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:15.224567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38b420-c170-4f76-b35d-e7200cf2a887 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:15.226207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 568/1152] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:56:15 2022] GET /identity/v3/projects/9e3d8392624e4378ac432e9fc6975af9 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:15.232262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:15.264489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:15.337384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24246be5-f268-4088-b78d-a42c2f5890ae None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:15.373573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fe86e-78d9-48b0-a739-937f80c7151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:15.374240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe86e-78d9-48b0-a739-937f80c7151e None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:15.374524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe86e-78d9-48b0-a739-937f80c7151e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:15.374838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe86e-78d9-48b0-a739-937f80c7151e None admin] PATH_INFO: `/v3/projects/9e3d8392624e4378ac432e9fc6975af9/users/bf73c7e47ef0423c9a781c2b5497cefc/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:15.376914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fe86e-78d9-48b0-a739-937f80c7151e None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3d8392624e4378ac432e9fc6975af9, user_id=bf73c7e47ef0423c9a781c2b5497cefc, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:15.378373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fe86e-78d9-48b0-a739-937f80c7151e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:15.393132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 585/1153] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:56:15 2022] PUT /identity/v3/projects/9e3d8392624e4378ac432e9fc6975af9/users/bf73c7e47ef0423c9a781c2b5497cefc/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:56:17.343433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8561-0a5a-406f-a9f6-6e984458a072 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:17.344093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8561-0a5a-406f-a9f6-6e984458a072 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:17.344093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a8561-0a5a-406f-a9f6-6e984458a072 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:17.345583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 569/1154] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:17.351696 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c52e-b15c-4dca-93e1-fd90ea5b6b47 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:17.352549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c52e-b15c-4dca-93e1-fd90ea5b6b47 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:17.352712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6c52e-b15c-4dca-93e1-fd90ea5b6b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:17.374254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-08e6c52e-b15c-4dca-93e1-fd90ea5b6b47 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:17.375509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-08e6c52e-b15c-4dca-93e1-fd90ea5b6b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:17.457085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 586/1155] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:17.726536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8561-0a5a-406f-a9f6-6e984458a072 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:17.834203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a8561-0a5a-406f-a9f6-6e984458a072 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:17.869413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:17.870052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:17.870335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:17.871305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:17.883611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:56:17.889400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:56:17.891259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:56:17.892207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 570/1156] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:56:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:56:17.897102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6c52e-b15c-4dca-93e1-fd90ea5b6b47 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:17.920250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:18.002731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6c52e-b15c-4dca-93e1-fd90ea5b6b47 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:18.040397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:18.041003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:18.041281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:18.041553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:18.042254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:18.043244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:18.044527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:18.053352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 587/1157] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:56:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:18.059500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:18.071148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:18.189385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fcfac-c297-4611-baa6-b5f6011710dd None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:18.224576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:18.225212 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:18.225496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:18.225770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] PATH_INFO: `/v3/users/bf73c7e47ef0423c9a781c2b5497cefc` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:18.226937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] RBAC: Authorizing `identity:get_user(user_id=bf73c7e47ef0423c9a781c2b5497cefc)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:18.228415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:18.230064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 571/1158] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:18 2022] GET /identity/v3/users/bf73c7e47ef0423c9a781c2b5497cefc => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:18.236411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:18.327413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:18.353806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232957e0-eca6-46ae-b916-c17d0053d2ff None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:18.387795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:18.388399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:18.388685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:18.388959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:18.463635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:18.465900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:18.552604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:18.578302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 588/1159] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:56:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:56:18.584332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:18.669333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:18.694174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9907d6b-a196-4882-91a3-dea98901a5e5 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:18.734646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07a7da-da2b-4ae8-9207-4a7d822bc6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:18.735313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07a7da-da2b-4ae8-9207-4a7d822bc6e8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:18.735595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07a7da-da2b-4ae8-9207-4a7d822bc6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:18.735874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07a7da-da2b-4ae8-9207-4a7d822bc6e8 None admin] PATH_INFO: `/v3/projects/9e3d8392624e4378ac432e9fc6975af9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:18.736991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07a7da-da2b-4ae8-9207-4a7d822bc6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=9e3d8392624e4378ac432e9fc6975af9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:18.738439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07a7da-da2b-4ae8-9207-4a7d822bc6e8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:18.740125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 572/1160] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:56:18 2022] GET /identity/v3/projects/9e3d8392624e4378ac432e9fc6975af9 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:18.748063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:18.853077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b516f-c739-4e33-873e-bfcaf27c1a4c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:18.887801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb7f2a-bd2a-4a92-b37d-88ab6f5a7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:18.888424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7f2a-bd2a-4a92-b37d-88ab6f5a7efc None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:18.888704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7f2a-bd2a-4a92-b37d-88ab6f5a7efc None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:18.888978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7f2a-bd2a-4a92-b37d-88ab6f5a7efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:18.889715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7f2a-bd2a-4a92-b37d-88ab6f5a7efc None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=9e3d8392624e4378ac432e9fc6975af9, user.id=bf73c7e47ef0423c9a781c2b5497cefc) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:18.890021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7f2a-bd2a-4a92-b37d-88ab6f5a7efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:18.891782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7f2a-bd2a-4a92-b37d-88ab6f5a7efc None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:18.898302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 589/1161] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:56:18 2022] GET /identity/v3/role_assignments?user.id=bf73c7e47ef0423c9a781c2b5497cefc&scope.project.id=9e3d8392624e4378ac432e9fc6975af9&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:20.863474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05f6b-8b5e-4e1f-968a-cc61857f752e None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:20.864060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05f6b-8b5e-4e1f-968a-cc61857f752e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:20.864060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05f6b-8b5e-4e1f-968a-cc61857f752e None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:20.865410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 573/1162] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:20.871032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe67c-9dde-4a7d-87fa-04fe6cec82f9 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:20.871560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe67c-9dde-4a7d-87fa-04fe6cec82f9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:20.871715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe67c-9dde-4a7d-87fa-04fe6cec82f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:20.890831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-19ebe67c-9dde-4a7d-87fa-04fe6cec82f9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:20.892078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-19ebe67c-9dde-4a7d-87fa-04fe6cec82f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:20.937391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:20.964063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 590/1163] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:21.241245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e05f6b-8b5e-4e1f-968a-cc61857f752e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:21.351138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e05f6b-8b5e-4e1f-968a-cc61857f752e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:21.396374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:21.396999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:21.397307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:21.397587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:21.414690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:21.473443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:21.474913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:21.587285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 574/1164] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:56:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:56:21.593882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebe67c-9dde-4a7d-87fa-04fe6cec82f9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:21.732274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebe67c-9dde-4a7d-87fa-04fe6cec82f9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:21.769552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505c2f0-8c4b-4ab3-aca8-51152089b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:21.770194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505c2f0-8c4b-4ab3-aca8-51152089b08a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:21.770477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505c2f0-8c4b-4ab3-aca8-51152089b08a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:21.770759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505c2f0-8c4b-4ab3-aca8-51152089b08a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:21.778641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505c2f0-8c4b-4ab3-aca8-51152089b08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:21.780124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505c2f0-8c4b-4ab3-aca8-51152089b08a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-8505c2f0-8c4b-4ab3-aca8-51152089b08a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:56:21.792006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:56:21.793878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:56:21.795035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 591/1165] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:56:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:56:21.800592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:21.852145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:21.918818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d2458-aad6-4f93-bd2d-3e5613c9430e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:21.954538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdafb4-d890-4fc2-aa99-893ebf85a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:21.955208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdafb4-d890-4fc2-aa99-893ebf85a631 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:21.955488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdafb4-d890-4fc2-aa99-893ebf85a631 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:21.955765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdafb4-d890-4fc2-aa99-893ebf85a631 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:21.956424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdafb4-d890-4fc2-aa99-893ebf85a631 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:21.956727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdafb4-d890-4fc2-aa99-893ebf85a631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:21.958110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdafb4-d890-4fc2-aa99-893ebf85a631 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:21.966995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 575/1166] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Aug 29 18:56:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:23.894751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce917bb-cf98-400e-ac7e-2ecc79facc99 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:23.895331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce917bb-cf98-400e-ac7e-2ecc79facc99 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:23.895331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce917bb-cf98-400e-ac7e-2ecc79facc99 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:23.896659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 592/1167] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:23.902196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06308b80-ae44-45a3-8b43-dfb17501ccfa None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:23.903049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06308b80-ae44-45a3-8b43-dfb17501ccfa None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:23.903049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06308b80-ae44-45a3-8b43-dfb17501ccfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:23.922388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-06308b80-ae44-45a3-8b43-dfb17501ccfa None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:23.923641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-06308b80-ae44-45a3-8b43-dfb17501ccfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:23.995969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 576/1168] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:24.280213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce917bb-cf98-400e-ac7e-2ecc79facc99 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:24.316456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:24.385034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce917bb-cf98-400e-ac7e-2ecc79facc99 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:24.424856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ddfe2-ae9d-48e5-8969-b648f0041503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:24.425472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ddfe2-ae9d-48e5-8969-b648f0041503 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:24.425754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ddfe2-ae9d-48e5-8969-b648f0041503 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:24.426030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ddfe2-ae9d-48e5-8969-b648f0041503 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:24.426709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ddfe2-ae9d-48e5-8969-b648f0041503 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:24.428124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ddfe2-ae9d-48e5-8969-b648f0041503 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:24.447809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 593/1169] 10.222.0.94 () {60 vars in 1138 bytes} [Mon Aug 29 18:56:24 2022] POST /identity/v3/domains => generated 267 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:56:26.440534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e57511-36f5-4254-bdc5-e1c856b36cb5 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:26.441096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e57511-36f5-4254-bdc5-e1c856b36cb5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:26.441096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e57511-36f5-4254-bdc5-e1c856b36cb5 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:26.442448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 577/1170] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:26.447986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5161-4f2a-4a0c-92b8-c03d0771179e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:26.448434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5161-4f2a-4a0c-92b8-c03d0771179e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:26.448528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5161-4f2a-4a0c-92b8-c03d0771179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:26.466920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c9ef5161-4f2a-4a0c-92b8-c03d0771179e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:26.468158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c9ef5161-4f2a-4a0c-92b8-c03d0771179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:26.550081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 594/1171] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:26.828195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e57511-36f5-4254-bdc5-e1c856b36cb5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:26.876468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:26.942155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e57511-36f5-4254-bdc5-e1c856b36cb5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:26.981698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36cac0-fc71-4603-85a4-47bae1bf92d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:26.982335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36cac0-fc71-4603-85a4-47bae1bf92d2 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:26.982612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36cac0-fc71-4603-85a4-47bae1bf92d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:26.982881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36cac0-fc71-4603-85a4-47bae1bf92d2 None admin] PATH_INFO: `/v3/domains/7141f84c8fbc448ca677d1ba2a9ae523` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:26.994295 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36cac0-fc71-4603-85a4-47bae1bf92d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7141f84c8fbc448ca677d1ba2a9ae523)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:26.995823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36cac0-fc71-4603-85a4-47bae1bf92d2 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:26.997755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 578/1172] 10.222.0.94 () {56 vars in 1183 bytes} [Mon Aug 29 18:56:26 2022] GET /identity/v3/domains/7141f84c8fbc448ca677d1ba2a9ae523 => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:56:27.003496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef5161-4f2a-4a0c-92b8-c03d0771179e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:27.033449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:27.117846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef5161-4f2a-4a0c-92b8-c03d0771179e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:27.155091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd568f-f888-4a3b-894d-4b776df1d863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:27.155734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd568f-f888-4a3b-894d-4b776df1d863 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:27.156014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd568f-f888-4a3b-894d-4b776df1d863 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:27.156287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd568f-f888-4a3b-894d-4b776df1d863 None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:27.156964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd568f-f888-4a3b-894d-4b776df1d863 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:27.158522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd568f-f888-4a3b-894d-4b776df1d863 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:27.188466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 595/1173] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:56:27 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:56:29.227383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce6b1-6c3f-4f98-b5ef-44c156b5c32e None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:29.227917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce6b1-6c3f-4f98-b5ef-44c156b5c32e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:29.228260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce6b1-6c3f-4f98-b5ef-44c156b5c32e None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:29.231230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 579/1174] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:29.237975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6576685-51da-4f63-a5fc-20d1c5d24d38 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:29.238536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6576685-51da-4f63-a5fc-20d1c5d24d38 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:29.238536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6576685-51da-4f63-a5fc-20d1c5d24d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:29.258679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d6576685-51da-4f63-a5fc-20d1c5d24d38 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:29.259955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d6576685-51da-4f63-a5fc-20d1c5d24d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:29.335607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 596/1175] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:29.622378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cce6b1-6c3f-4f98-b5ef-44c156b5c32e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:29.746796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:29.752152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cce6b1-6c3f-4f98-b5ef-44c156b5c32e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:29.789283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e1476-35b9-4ffa-ae3e-d18c424d1df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:29.789917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e1476-35b9-4ffa-ae3e-d18c424d1df5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:29.790195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e1476-35b9-4ffa-ae3e-d18c424d1df5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:29.790466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e1476-35b9-4ffa-ae3e-d18c424d1df5 None admin] PATH_INFO: `/v3/domains/7141f84c8fbc448ca677d1ba2a9ae523` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:29.791638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e1476-35b9-4ffa-ae3e-d18c424d1df5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7141f84c8fbc448ca677d1ba2a9ae523)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:29.793267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e1476-35b9-4ffa-ae3e-d18c424d1df5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:29.794902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 580/1176] 10.222.0.94 () {56 vars in 1183 bytes} [Mon Aug 29 18:56:29 2022] GET /identity/v3/domains/7141f84c8fbc448ca677d1ba2a9ae523 => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:29.800963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6576685-51da-4f63-a5fc-20d1c5d24d38 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:29.812411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:29.915610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6576685-51da-4f63-a5fc-20d1c5d24d38 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:29.959955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d0414-04fd-487b-aae8-76e0ed385eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:29.960932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d0414-04fd-487b-aae8-76e0ed385eff None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:29.961384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d0414-04fd-487b-aae8-76e0ed385eff None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:29.961833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d0414-04fd-487b-aae8-76e0ed385eff None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:29.962963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d0414-04fd-487b-aae8-76e0ed385eff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:29.965581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d0414-04fd-487b-aae8-76e0ed385eff None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:29.995556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 597/1177] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:56:29 2022] POST /identity/v3/users => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:56:31.991447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfaca5-7b2f-4aa0-8dd8-1619e6885ba5 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:31.992000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfaca5-7b2f-4aa0-8dd8-1619e6885ba5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:31.992000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfaca5-7b2f-4aa0-8dd8-1619e6885ba5 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:31.993367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 581/1178] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:31.999220 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b251e-c6e4-4ff7-8716-0d8632074e3c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:31.999747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b251e-c6e4-4ff7-8716-0d8632074e3c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:31.999747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b251e-c6e4-4ff7-8716-0d8632074e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:32.022776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b89b251e-c6e4-4ff7-8716-0d8632074e3c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:32.024022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b89b251e-c6e4-4ff7-8716-0d8632074e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:32.086849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:32.099822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 598/1179] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:32.382891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfaca5-7b2f-4aa0-8dd8-1619e6885ba5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:32.493682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfaca5-7b2f-4aa0-8dd8-1619e6885ba5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:32.530274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463db796-42d0-422a-a761-1f6013381fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:32.531023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463db796-42d0-422a-a761-1f6013381fff None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:32.531368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463db796-42d0-422a-a761-1f6013381fff None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:32.531643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463db796-42d0-422a-a761-1f6013381fff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:32.542657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463db796-42d0-422a-a761-1f6013381fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-463db796-42d0-422a-a761-1f6013381fff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:56:32.548222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:56:32.549904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:56:32.550816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 582/1180] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:56:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:56:32.555615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b251e-c6e4-4ff7-8716-0d8632074e3c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:32.664027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b251e-c6e4-4ff7-8716-0d8632074e3c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:32.702569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:32.703371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:32.703649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:32.703919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:32.704598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:32.704893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:32.706267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:32.716154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 599/1181] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:56:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:32.722432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463db796-42d0-422a-a761-1f6013381fff None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:32.754097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:32.834535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463db796-42d0-422a-a761-1f6013381fff None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:32.872909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:32.873684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:32.874007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:32.874286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] PATH_INFO: `/v3/users/59cdc5dcd05b4ff386dd495df5522429` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:32.889388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] RBAC: Authorizing `identity:get_user(user_id=59cdc5dcd05b4ff386dd495df5522429)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:32.890834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:32.892824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 583/1182] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:32 2022] GET /identity/v3/users/59cdc5dcd05b4ff386dd495df5522429 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:32.901305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:32.981421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:33.029478 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c62e6-f585-462e-a50f-d2feee2e522c None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:33.065067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:33.065681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:33.065969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:33.066246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:33.149818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:33.151352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:33.209041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:33.256089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 600/1183] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:56:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:56:33.263229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:33.368971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:33.387223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5733b7-8604-4702-b0d8-cf659b1d0105 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:33.424554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664daec-1a2f-4949-95e6-f60175e1fb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:33.425194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664daec-1a2f-4949-95e6-f60175e1fb5a None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:33.425479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664daec-1a2f-4949-95e6-f60175e1fb5a None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:33.425754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664daec-1a2f-4949-95e6-f60175e1fb5a None admin] PATH_INFO: `/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:33.427099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664daec-1a2f-4949-95e6-f60175e1fb5a None admin] RBAC: Authorizing `identity:get_project(project_id=a77f4f980fdd4aa8bc9603cf4b3775bb)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:33.428529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664daec-1a2f-4949-95e6-f60175e1fb5a None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:33.430225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 584/1184] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:56:33 2022] GET /identity/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb => generated 361 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:33.438964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:33.559704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e336703-f019-48ab-8a1a-6076558c5440 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:33.595218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc96b0-a0c8-487b-9e50-61371fdefba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:33.595828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc96b0-a0c8-487b-9e50-61371fdefba9 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:33.596104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc96b0-a0c8-487b-9e50-61371fdefba9 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:33.596374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc96b0-a0c8-487b-9e50-61371fdefba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:33.597094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc96b0-a0c8-487b-9e50-61371fdefba9 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=a77f4f980fdd4aa8bc9603cf4b3775bb, user.id=59cdc5dcd05b4ff386dd495df5522429) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:33.597398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc96b0-a0c8-487b-9e50-61371fdefba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:33.598813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc96b0-a0c8-487b-9e50-61371fdefba9 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:33.604538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 601/1185] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:56:33 2022] GET /identity/v3/role_assignments?user.id=59cdc5dcd05b4ff386dd495df5522429&scope.project.id=a77f4f980fdd4aa8bc9603cf4b3775bb&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:35.583684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd5c60-39cd-41a4-bb0c-373ed079f9d1 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:35.584256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd5c60-39cd-41a4-bb0c-373ed079f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:35.584256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd5c60-39cd-41a4-bb0c-373ed079f9d1 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:35.586603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 585/1186] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:35.594676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3efd-6d29-45ab-ad22-a948a466df32 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:35.595234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3efd-6d29-45ab-ad22-a948a466df32 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:35.595372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3efd-6d29-45ab-ad22-a948a466df32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:35.615919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-90ee3efd-6d29-45ab-ad22-a948a466df32 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:35.617870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-90ee3efd-6d29-45ab-ad22-a948a466df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:35.635554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:35.694694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 602/1187] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:56:36.193009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd5c60-39cd-41a4-bb0c-373ed079f9d1 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:36.304837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd5c60-39cd-41a4-bb0c-373ed079f9d1 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:36.341208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:36.341851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:36.342131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:36.342536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:36.353469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:56:36.358571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:56:36.359742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:56:36.360968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 586/1188] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:56:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:56:36.367623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee3efd-6d29-45ab-ad22-a948a466df32 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:36.491948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee3efd-6d29-45ab-ad22-a948a466df32 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:36.536613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:36.537602 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:36.538064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:36.538505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:36.539574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:36.540074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:36.542485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:36.557580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 603/1189] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:56:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:36.564711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:36.581194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:36.672899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434198-88fc-46d8-948b-88843f263250 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:36.709346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:36.709977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:36.710260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:36.710539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] PATH_INFO: `/v3/users/59cdc5dcd05b4ff386dd495df5522429` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:36.711703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] RBAC: Authorizing `identity:get_user(user_id=59cdc5dcd05b4ff386dd495df5522429)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:36.713143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:36.716010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 587/1190] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:36 2022] GET /identity/v3/users/59cdc5dcd05b4ff386dd495df5522429 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:36.722145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:36.739867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:36.834352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7d4a9-7917-4f12-8575-1f80abf7740d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:36.873119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461eb60-d100-46d9-bdb9-4014bbb2af08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:36.874116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461eb60-d100-46d9-bdb9-4014bbb2af08 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:36.874574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461eb60-d100-46d9-bdb9-4014bbb2af08 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:36.875026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461eb60-d100-46d9-bdb9-4014bbb2af08 None admin] PATH_INFO: `/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:36.876847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461eb60-d100-46d9-bdb9-4014bbb2af08 None admin] RBAC: Authorizing `identity:get_project(project_id=a77f4f980fdd4aa8bc9603cf4b3775bb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:36.879358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461eb60-d100-46d9-bdb9-4014bbb2af08 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:36.882169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 604/1191] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:56:36 2022] GET /identity/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:36.888945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:36.981795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:37.002126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68be762-f468-48a9-b260-0ae959a6de97 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:37.040720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e44d8-e502-4dfc-bab9-6c42f164e6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:37.041331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e44d8-e502-4dfc-bab9-6c42f164e6a1 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:37.041617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e44d8-e502-4dfc-bab9-6c42f164e6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:37.041889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e44d8-e502-4dfc-bab9-6c42f164e6a1 None admin] PATH_INFO: `/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb/users/59cdc5dcd05b4ff386dd495df5522429/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:37.044523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e44d8-e502-4dfc-bab9-6c42f164e6a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a77f4f980fdd4aa8bc9603cf4b3775bb, user_id=59cdc5dcd05b4ff386dd495df5522429, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:37.045970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e44d8-e502-4dfc-bab9-6c42f164e6a1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:37.060381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 588/1192] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:56:36 2022] PUT /identity/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb/users/59cdc5dcd05b4ff386dd495df5522429/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:56:39.061957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c456-87b0-41ac-ab4d-6ea1ce190d6d None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:39.062485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c456-87b0-41ac-ab4d-6ea1ce190d6d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:39.062485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c456-87b0-41ac-ab4d-6ea1ce190d6d None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:39.063878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 605/1193] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:56:39.069561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438c71d-0fc9-4f51-8832-6b61789ba518 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:39.070210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438c71d-0fc9-4f51-8832-6b61789ba518 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:39.070210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438c71d-0fc9-4f51-8832-6b61789ba518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:39.088616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c438c71d-0fc9-4f51-8832-6b61789ba518 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:56:39.090278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c438c71d-0fc9-4f51-8832-6b61789ba518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:56:39.167378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 589/1194] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:56:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:56:39.446401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4c456-87b0-41ac-ab4d-6ea1ce190d6d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:39.535823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:39.556732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4c456-87b0-41ac-ab4d-6ea1ce190d6d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:39.593495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:39.594125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:39.594402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:39.594668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:39.605200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:56:39.610058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:56:39.611285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:56:39.612199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 606/1195] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:56:39.617613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438c71d-0fc9-4f51-8832-6b61789ba518 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:39.691914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:39.730252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438c71d-0fc9-4f51-8832-6b61789ba518 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:39.781584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:39.782201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:39.782473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:39.782746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:39.783450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:39.783748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:39.785098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:39.794736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 590/1196] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:39.802627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:39.920033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740300f8-ae63-4f7b-9455-23129a2aba69 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:39.961669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:39.962296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:39.962597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:39.962867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] PATH_INFO: `/v3/users/59cdc5dcd05b4ff386dd495df5522429` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:39.964017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] RBAC: Authorizing `identity:get_user(user_id=59cdc5dcd05b4ff386dd495df5522429)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:39.965829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:39.967493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 607/1197] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Aug 29 18:56:39 2022] GET /identity/v3/users/59cdc5dcd05b4ff386dd495df5522429 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:39.973875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:40.082268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924439-b02f-4675-bfa8-0e454648fed7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:40.118323 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:40.125101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:40.125991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:40.126137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:40.126391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:40.207786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:40.209259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:40.321446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 591/1198] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:56:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:56:40.328200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:40.341618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:40.438723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d3192-7cf1-471c-90b6-6d0dd26eed9a None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:40.473879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc761c1b-4827-4732-91d0-31f9a1ea7891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:40.474506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761c1b-4827-4732-91d0-31f9a1ea7891 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:40.474783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761c1b-4827-4732-91d0-31f9a1ea7891 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:40.475055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761c1b-4827-4732-91d0-31f9a1ea7891 None admin] PATH_INFO: `/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:40.476186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc761c1b-4827-4732-91d0-31f9a1ea7891 None admin] RBAC: Authorizing `identity:get_project(project_id=a77f4f980fdd4aa8bc9603cf4b3775bb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:40.477614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc761c1b-4827-4732-91d0-31f9a1ea7891 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:40.479264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 608/1199] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Aug 29 18:56:40 2022] GET /identity/v3/projects/a77f4f980fdd4aa8bc9603cf4b3775bb => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:56:40.486601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:56:40.520195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:40.593504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd3f23-dd57-42c6-91e3-4770e280f60d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:56:40.632533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95cf04-425c-4aaf-abd7-67246bc2213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:56:40.633252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95cf04-425c-4aaf-abd7-67246bc2213b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:56:40.633530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95cf04-425c-4aaf-abd7-67246bc2213b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:56:40.633878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95cf04-425c-4aaf-abd7-67246bc2213b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:56:40.634772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95cf04-425c-4aaf-abd7-67246bc2213b None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=a77f4f980fdd4aa8bc9603cf4b3775bb, user.id=59cdc5dcd05b4ff386dd495df5522429) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:56:40.635094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95cf04-425c-4aaf-abd7-67246bc2213b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:56:40.636536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95cf04-425c-4aaf-abd7-67246bc2213b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:56:40.643357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 592/1200] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:56:40 2022] GET /identity/v3/role_assignments?user.id=59cdc5dcd05b4ff386dd495df5522429&scope.project.id=a77f4f980fdd4aa8bc9603cf4b3775bb&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:56:50.469664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=62 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:56:50.641062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=52 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:38.804738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5ae50-8271-4dc7-9c9f-77cb89d96a02 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.805342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5ae50-8271-4dc7-9c9f-77cb89d96a02 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.805342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5ae50-8271-4dc7-9c9f-77cb89d96a02 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.807091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 609/1201] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:38.813773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f93de-62b9-41f8-96cd-cbab8d92ab51 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:38.814287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f93de-62b9-41f8-96cd-cbab8d92ab51 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:38.814406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f93de-62b9-41f8-96cd-cbab8d92ab51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:38.846857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-026f93de-62b9-41f8-96cd-cbab8d92ab51 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:38.848181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-026f93de-62b9-41f8-96cd-cbab8d92ab51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:38.951262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 593/1202] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:57:39.336555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5ae50-8271-4dc7-9c9f-77cb89d96a02 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.509388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5ae50-8271-4dc7-9c9f-77cb89d96a02 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.558589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.559345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.559643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.559919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.573646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:39.575360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:39.593461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:39.594575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:39.595833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:39.595833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:39.595833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:39.599472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 610/1203] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:39.603387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f93de-62b9-41f8-96cd-cbab8d92ab51 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.750379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f93de-62b9-41f8-96cd-cbab8d92ab51 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:39.798300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:39.812407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93950118-3685-4613-855c-70eaee3ab08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:39.813223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950118-3685-4613-855c-70eaee3ab08b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:39.813542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950118-3685-4613-855c-70eaee3ab08b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:39.813821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950118-3685-4613-855c-70eaee3ab08b None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:39.814509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93950118-3685-4613-855c-70eaee3ab08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:39.814819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93950118-3685-4613-855c-70eaee3ab08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:39.817336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93950118-3685-4613-855c-70eaee3ab08b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:39.843126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 594/1204] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:39.844560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:39.939982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:40.018279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61709776-1577-4f27-884e-ed8cab310769 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:40.064274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7d1fe-e543-49f9-a3c3-35a6f11b78eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:40.064998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7d1fe-e543-49f9-a3c3-35a6f11b78eb None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:40.065475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7d1fe-e543-49f9-a3c3-35a6f11b78eb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:40.065841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7d1fe-e543-49f9-a3c3-35a6f11b78eb None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:40.066522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7d1fe-e543-49f9-a3c3-35a6f11b78eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:40.068045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7d1fe-e543-49f9-a3c3-35a6f11b78eb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:40.094127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 611/1205] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Aug 29 18:57:39 2022] POST /identity/v3/users => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:57:42.570756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cc49b-a01f-498d-9523-33ee049c0ee2 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.571341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cc49b-a01f-498d-9523-33ee049c0ee2 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.571432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cc49b-a01f-498d-9523-33ee049c0ee2 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.573107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 595/1206] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:57:42.579825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f2a71-142a-4cf9-a0b7-04aa98bd0361 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:42.580775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f2a71-142a-4cf9-a0b7-04aa98bd0361 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:42.580775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f2a71-142a-4cf9-a0b7-04aa98bd0361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:42.609529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-539f2a71-142a-4cf9-a0b7-04aa98bd0361 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:42.610929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-539f2a71-142a-4cf9-a0b7-04aa98bd0361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:42.691287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 612/1207] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:57:43.022959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cc49b-a01f-498d-9523-33ee049c0ee2 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:43.200515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cc49b-a01f-498d-9523-33ee049c0ee2 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:43.250276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:43.250948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.251242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.251521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.264938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:43.270571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:43.271849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:43.273783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 596/1208] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:43.280325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f2a71-142a-4cf9-a0b7-04aa98bd0361 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:43.344297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.423013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f2a71-142a-4cf9-a0b7-04aa98bd0361 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:43.472630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:43.473269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.473612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.473889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.474601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:43.474936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:43.476435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:43.491421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 613/1209] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:43.493239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:43.564368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.625703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5f5a8-d56f-4843-bf30-2325b738b4cd None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:43.667161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:43.667886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.668163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.668433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.681737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:43.683266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:43.698876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:43.700343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:43.702597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:43.702597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:43.702597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:43.705109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 597/1210] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:43.707759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:43.858852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:43.863193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2b458-b07e-4af2-a88b-b86e230ab63e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:43.916392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:43.917068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:43.917349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:43.917624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:43.918314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:43.918704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:43.920805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:43.940974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 614/1211] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:43.946735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.093436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e5be-ef1a-4ccd-b17b-4beb04b12fbf None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.094778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.142967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.143642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.143919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.144190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] PATH_INFO: `/v3/users/placement` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.153194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.155327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:57:44.168658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:44.171361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:44.172623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 598/1212] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:44.177175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.301377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b0706-f998-4e88-8974-64cf8d5a6c34 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.360420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.361367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.361858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.362345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.363718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:44.364268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.367327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.395491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 615/1213] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Aug 29 18:57:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:44.402693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.562590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57f65f-4119-40a2-8742-ee8732036c8e None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.605580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.606341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.606645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.606912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:44.642629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.688148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:44.689602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:44.807542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 599/1214] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:57:44.814073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:44.842027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:44.947679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f12e2-2fd2-43c3-b4ad-c395f82288d7 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:44.992636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:44.993626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:44.993840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:44.994135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.006227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.007844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:45.025216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:45.031946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:45.033207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:45.033207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:45.033207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:45.033207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 616/1215] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:45.033498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.110140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.157753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e922c-4c3f-424f-9feb-d3bed6d501f0 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.205608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.206221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.206558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.206844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.207672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:45.208034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.209504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.223284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 600/1216] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:45.229136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.308041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.354912 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133e3b2-9da6-4de5-b334-bc73cb87d8e0 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.398072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.398834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.399145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.399424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.407387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.408896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:57:45.422164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:45.423694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:45.425017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 617/1217] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:45.430386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.564978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9a292-b5ff-4b7e-a00a-1249ec510b8b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.592298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.606467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeae96-843d-4b49-93b7-7f1f3ea3c6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.607134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeae96-843d-4b49-93b7-7f1f3ea3c6b1 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.607951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeae96-843d-4b49-93b7-7f1f3ea3c6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.608234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeae96-843d-4b49-93b7-7f1f3ea3c6b1 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.609005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeae96-843d-4b49-93b7-7f1f3ea3c6b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:45.609255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeae96-843d-4b49-93b7-7f1f3ea3c6b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.610698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeae96-843d-4b49-93b7-7f1f3ea3c6b1 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.624453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 601/1218] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:45.631939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:45.761961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4b6e2-bb49-427f-bbbb-f2d78b17f479 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:45.792908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:45.801517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b96174-d2d7-4018-b129-ec8d0f99beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:45.802283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96174-d2d7-4018-b129-ec8d0f99beeb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:45.804658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96174-d2d7-4018-b129-ec8d0f99beeb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:45.804658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96174-d2d7-4018-b129-ec8d0f99beeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:45.804658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96174-d2d7-4018-b129-ec8d0f99beeb None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=005b6400ae0c40bd8da5b9ddcc96770b) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:45.804658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96174-d2d7-4018-b129-ec8d0f99beeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:45.805403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96174-d2d7-4018-b129-ec8d0f99beeb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:45.811835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 618/1219] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:45 2022] GET /identity/v3/role_assignments?user.id=005b6400ae0c40bd8da5b9ddcc96770b&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:47.944993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46edae0-bc01-450d-ad29-81eb147969c3 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.945541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46edae0-bc01-450d-ad29-81eb147969c3 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.945541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46edae0-bc01-450d-ad29-81eb147969c3 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.947139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 602/1220] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:47.952606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ad821-beaf-4683-bc51-68bb721abe60 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:47.953104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ad821-beaf-4683-bc51-68bb721abe60 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:47.953202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ad821-beaf-4683-bc51-68bb721abe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:47.972691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4a5ad821-beaf-4683-bc51-68bb721abe60 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:47.973908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4a5ad821-beaf-4683-bc51-68bb721abe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:48.051029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 619/1221] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:57:48.361124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46edae0-bc01-450d-ad29-81eb147969c3 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.485251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46edae0-bc01-450d-ad29-81eb147969c3 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.526318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.527248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.527693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.528084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.542622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:48.549113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:48.550445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:48.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 603/1222] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:48.558183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ad821-beaf-4683-bc51-68bb721abe60 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.679854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ad821-beaf-4683-bc51-68bb721abe60 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.707533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:48.720611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.721224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.721502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.721774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.722436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:48.722738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:48.724141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:48.734101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 620/1223] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:48.740203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:48.791956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:48.860454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d48a-7058-4902-80d1-0f43b727a1ea None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:48.900643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:48.901281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:48.901563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:48.901820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:48.913033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:48.914550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:48.932727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:48.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:48.934962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:48.934962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:48.934962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:48.934962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 604/1224] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:48.943579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:49.068976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad5313-bb35-4929-bab9-0e06cd32e5c8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:49.115525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:49.116437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.116821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.117189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.118038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:49.118490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:49.120221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:49.140925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 621/1225] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:49.147188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:49.269011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:49.278434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e8081-5eb8-4c67-93ba-a35e85575aee None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:49.320869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:49.321890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.322366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.322834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.336707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:49.338199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:57:49.351679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:49.352820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:49.354092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 605/1226] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:49.359102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:49.416151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:49.485923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63d4de-fd0f-4603-883b-2f25d9edbf25 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:49.525717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:49.526332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.526766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.527140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.528140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:49.528472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:49.531550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:49.552636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 622/1227] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Aug 29 18:57:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:49.558346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:49.681651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2a7b-baff-416c-addb-954c792a10a4 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:49.729208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:49.729894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.730171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.730520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.744166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:49.745644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:49.762531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:49.764697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:49.765645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:49.765645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:49.765645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:49.765645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 606/1228] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:49.771189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:49.897877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7caea-0e80-4a09-8b66-042bb5f6e169 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:49.900657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:49.939275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:49.939899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:49.940178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:49.940458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:49.941187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:49.941577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:49.943207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:49.959624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 623/1229] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:49.965991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:49.979567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:50.105884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac840acd-379b-487b-ab1b-4add359a861c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:50.152626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:50.153372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.153752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.154398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.162590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:50.164313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:57:50.184064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:50.189256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:57:50.191288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:50.192957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 607/1230] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:50.200017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:50.370171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410acf7-57ca-4a8f-825f-cffe5871791e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:50.422013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859a718-d8c0-470d-887f-3c6fb24f70ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:50.422727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859a718-d8c0-470d-887f-3c6fb24f70ee None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.423009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859a718-d8c0-470d-887f-3c6fb24f70ee None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.423320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859a718-d8c0-470d-887f-3c6fb24f70ee None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.424112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859a718-d8c0-470d-887f-3c6fb24f70ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:50.424427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859a718-d8c0-470d-887f-3c6fb24f70ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:50.426189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859a718-d8c0-470d-887f-3c6fb24f70ee None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:50.441412 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 624/1231] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:57:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:50.447929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:50.528855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:50.584659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e9a3e-03a8-47d4-8a90-77b44b91b9ba None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:50.628776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a58476-dc56-49bd-8ff5-984f5d1f7242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:50.629361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58476-dc56-49bd-8ff5-984f5d1f7242 None admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:50.629643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58476-dc56-49bd-8ff5-984f5d1f7242 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:50.629920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58476-dc56-49bd-8ff5-984f5d1f7242 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/005b6400ae0c40bd8da5b9ddcc96770b/roles/6328cebc97c34f12bc2c15bafa4ba1e0` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:50.647938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a58476-dc56-49bd-8ff5-984f5d1f7242 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=005b6400ae0c40bd8da5b9ddcc96770b, role_id=6328cebc97c34f12bc2c15bafa4ba1e0)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:50.649397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a58476-dc56-49bd-8ff5-984f5d1f7242 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:50.664153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 608/1232] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:57:50 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/005b6400ae0c40bd8da5b9ddcc96770b/roles/6328cebc97c34f12bc2c15bafa4ba1e0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 18:57:52.713860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e234-23dd-4d77-88bd-578d70825adb None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.714470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e234-23dd-4d77-88bd-578d70825adb None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.714713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e234-23dd-4d77-88bd-578d70825adb None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.717401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 625/1233] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:57:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:52.726416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e470b0d-1901-4cf7-8560-1448ce00cc12 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:52.727178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e470b0d-1901-4cf7-8560-1448ce00cc12 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:52.727178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e470b0d-1901-4cf7-8560-1448ce00cc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:52.749805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5e470b0d-1901-4cf7-8560-1448ce00cc12 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:52.751271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e470b0d-1901-4cf7-8560-1448ce00cc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:52.846843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 609/1234] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:57:53.140551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e234-23dd-4d77-88bd-578d70825adb None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.173750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.304267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e234-23dd-4d77-88bd-578d70825adb None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.364380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.365428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.365924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.366321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.382443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:53.389488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:53.391517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:53.392802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 626/1235] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:53.398890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e470b0d-1901-4cf7-8560-1448ce00cc12 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.466509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.538274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e470b0d-1901-4cf7-8560-1448ce00cc12 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.586661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.586661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.586661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.587193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.587868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:53.588229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.589723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.605723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 610/1236] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:53.614169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:53.743128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:53.772419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241690c5-b13e-4df1-b354-47906be2d460 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:53.823405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:53.823965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:53.824283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:53.824532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:53.838606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:53.840283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:53.862231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:53.863348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:53.864267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:53.864267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:53.864267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:53.864267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 627/1237] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:53.870990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.030376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8b023-9ec1-4f2f-8730-2259b31d73d8 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.041499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.079722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.080513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.080826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.081233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.082175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:54.082616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.084282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.102446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 611/1238] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:54.109664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.285509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87e543-5b95-4243-8500-d3339c0241f0 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.321115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.321747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.322035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.322300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] PATH_INFO: `/v3/users/placement` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.325388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.332592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.334206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:57:54.347028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:54.348910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:54.349948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 628/1239] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:54.354652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.478763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a10bc-d2dd-4694-b803-88dee07133a7 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.521707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.522334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.522615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.522888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.523672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:54.523977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.525416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.546842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 612/1240] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:54.552977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.677133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77521f5c-178c-47de-a67d-c5f9bfbfc55f None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:54.730795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:54.731667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:54.732162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:54.732676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:54.822073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:54.831337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:54.833348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:54.958861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 629/1241] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:57:54.964921 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:54.999384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.087560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f63ee-e9d9-4f13-b512-09553e1873c3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.128605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.129345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.129678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.129957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.142471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.144205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:55.163320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:55.164480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:55.165631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:55.165631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:55.165631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:55.165631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 613/1242] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:55.171339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.301931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.308591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa946-3bc9-42c0-931b-a2260fcec033 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.349626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.349883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.349981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.350220 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.350869 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:55.351223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.352592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.365462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 630/1243] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:55.371427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.463045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:55.490942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c7408-0e24-45de-8011-fd29d72df189 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.537209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.538168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.538655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.539020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.551353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.553233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:57:55.568034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:55.569832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:57:55.571427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 614/1244] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:55.576345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.715885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8b28-0537-41a2-a4d8-b1babd0c3a9e None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.756467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8290d5-6ed9-42dd-9eff-98f80647f4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.757250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8290d5-6ed9-42dd-9eff-98f80647f4fc None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.757622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8290d5-6ed9-42dd-9eff-98f80647f4fc None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.757933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8290d5-6ed9-42dd-9eff-98f80647f4fc None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.758771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8290d5-6ed9-42dd-9eff-98f80647f4fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:55.759116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8290d5-6ed9-42dd-9eff-98f80647f4fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.760587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8290d5-6ed9-42dd-9eff-98f80647f4fc None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.775361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 631/1245] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:55.786201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:55.912571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38ac2f-a878-4643-a7fd-fd2e57908408 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:55.954997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49a459-f42e-4650-a6cc-22b7f813670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:55.955905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49a459-f42e-4650-a6cc-22b7f813670b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:55.956390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49a459-f42e-4650-a6cc-22b7f813670b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:55.956873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49a459-f42e-4650-a6cc-22b7f813670b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:55.958133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49a459-f42e-4650-a6cc-22b7f813670b None admin] RBAC: Adding query filter params (role.id=6328cebc97c34f12bc2c15bafa4ba1e0, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=005b6400ae0c40bd8da5b9ddcc96770b) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:55.958690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49a459-f42e-4650-a6cc-22b7f813670b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:55.961129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49a459-f42e-4650-a6cc-22b7f813670b None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:55.971032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 615/1246] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:57:55 2022] GET /identity/v3/role_assignments?user.id=005b6400ae0c40bd8da5b9ddcc96770b&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=6328cebc97c34f12bc2c15bafa4ba1e0 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:58.262893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c105f3-9e13-4aa2-af9b-857c31fb397a None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.263502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c105f3-9e13-4aa2-af9b-857c31fb397a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.263502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c105f3-9e13-4aa2-af9b-857c31fb397a None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.266274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 632/1247] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:57:58.271260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46861-04fb-4a41-b263-c089bf48ec21 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.271972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46861-04fb-4a41-b263-c089bf48ec21 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.271972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46861-04fb-4a41-b263-c089bf48ec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.279866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.296195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e1b46861-04fb-4a41-b263-c089bf48ec21 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:57:58.297430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e1b46861-04fb-4a41-b263-c089bf48ec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:57:58.383567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 616/1248] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:57:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:57:58.659214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c105f3-9e13-4aa2-af9b-857c31fb397a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:58.684082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:58.790530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c105f3-9e13-4aa2-af9b-857c31fb397a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:58.838516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:58.838516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:58.838516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:58.838516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:58.850758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:57:58.856467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:58.857764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:58.859014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 633/1249] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:57:58.864528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46861-04fb-4a41-b263-c089bf48ec21 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.000544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46861-04fb-4a41-b263-c089bf48ec21 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.036847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.050282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.050983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.051360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.052279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.052482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:59.052796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.054218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.065599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 617/1250] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:57:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.071959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.188356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.229006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2a9ad-ef05-422b-b012-0501bb1c270d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.277060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.277840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.278184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.278472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.292932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.295806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:57:59.319167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:59.326214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:57:59.328107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:57:59.328107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:57:59.328107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:59.328107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 634/1251] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:59.328538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.468874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54595479-0b5d-404e-8915-45fb094d8301 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.508850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.509461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.509812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.510129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.511789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:59.511789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.512696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.531168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 618/1252] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:57:59.538239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.687233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db9bd-6300-412c-9430-793cbc07f357 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.725213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.736258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.737208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.737666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.738073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] PATH_INFO: `/v3/users/placement` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.751168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.752806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:57:59.775265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:57:59.776388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:57:59.777335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 635/1253] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:57:59.784866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:57:59.853637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:57:59.910262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646eec8-1106-4613-824b-f093f8e9827c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:57:59.949309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:57:59.949999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:57:59.950314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:57:59.950725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:57:59.951606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:57:59.951913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:57:59.953544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:57:59.982272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 619/1254] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:57:59.990500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:00.153676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886406-d5c9-41a6-b21f-ed30531fb37d None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:00.210144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:00.210945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.211251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.211524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.284046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:00.300700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:00.302481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:00.424893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 636/1255] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:57:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:58:00.431884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:00.556269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eef01-644d-4eb8-9bfe-cc3110f4935c None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:00.583476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:00.609087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:00.609736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.611766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.611766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.624377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:00.625906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:00.644619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:00.645860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:00.647008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:00.647008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:00.647008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:00.647008 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 620/1256] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:00.652034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:00.766307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:00.785240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497364a1-8bbf-4241-a1f2-e2aff2bf5502 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:00.837219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:00.838001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:00.838292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:00.838662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:00.839223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:00.839534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:00.841059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:00.855476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 637/1257] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:00.862037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:00.997394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f83f0-e159-424a-9096-4f2dad21b1d4 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.041005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.042196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.042709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.043299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] PATH_INFO: `/v3/projects/service` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.056679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:01.059592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:01.085135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:01.086403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:01.087505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:01.088127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 621/1258] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:01.103337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.281471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1264a0-c878-4e67-848d-936c5a7fb2d2 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.335101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380ecdc-bb85-47d1-a765-dd989bd2fc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.335822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ecdc-bb85-47d1-a765-dd989bd2fc9b None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.336003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ecdc-bb85-47d1-a765-dd989bd2fc9b None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.336280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ecdc-bb85-47d1-a765-dd989bd2fc9b None admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.336986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ecdc-bb85-47d1-a765-dd989bd2fc9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:01.337329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ecdc-bb85-47d1-a765-dd989bd2fc9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:01.338796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ecdc-bb85-47d1-a765-dd989bd2fc9b None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:01.359387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 638/1259] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:01.360598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:01.390287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:01.485825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ca42a-492a-41fa-a42f-d3a17e3d9ccf None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:01.523681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f3bcb-b372-45a3-8dd5-44a1cca3d607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:01.524299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f3bcb-b372-45a3-8dd5-44a1cca3d607 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:01.524580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f3bcb-b372-45a3-8dd5-44a1cca3d607 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:01.524855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f3bcb-b372-45a3-8dd5-44a1cca3d607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:01.525590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f3bcb-b372-45a3-8dd5-44a1cca3d607 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=005b6400ae0c40bd8da5b9ddcc96770b) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:01.525903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f3bcb-b372-45a3-8dd5-44a1cca3d607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:01.527362 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f3bcb-b372-45a3-8dd5-44a1cca3d607 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:01.533882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 622/1260] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:01 2022] GET /identity/v3/role_assignments?user.id=005b6400ae0c40bd8da5b9ddcc96770b&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:03.495489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec522a6-16fc-4796-8b93-0c4d13f1eb52 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.496046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec522a6-16fc-4796-8b93-0c4d13f1eb52 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.496046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec522a6-16fc-4796-8b93-0c4d13f1eb52 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.497395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 639/1261] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:03.503856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d021b3-ac64-4fd8-8ccb-1692b2b9aca6 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:03.504385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d021b3-ac64-4fd8-8ccb-1692b2b9aca6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:03.504385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d021b3-ac64-4fd8-8ccb-1692b2b9aca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:03.523360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-01d021b3-ac64-4fd8-8ccb-1692b2b9aca6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:03.524584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-01d021b3-ac64-4fd8-8ccb-1692b2b9aca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:03.597269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 623/1262] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:58:03.867508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec522a6-16fc-4796-8b93-0c4d13f1eb52 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:03.879012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:03.985650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec522a6-16fc-4796-8b93-0c4d13f1eb52 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.020770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.021652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.021925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.022196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.033321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:04.038086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:04.039370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:58:04.040237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 640/1263] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:04.045377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d021b3-ac64-4fd8-8ccb-1692b2b9aca6 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.064089 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.149712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d021b3-ac64-4fd8-8ccb-1692b2b9aca6 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.190465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.191110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.191399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.191692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.192528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:04.192790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.194142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.208339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 624/1264] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:04.215866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.289294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.332978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae793f-5e68-4979-ba9f-bd91892e97d8 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.372175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.372823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.373113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.373917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.387534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.389160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:04.406093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:04.407144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:04.408481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:04.408481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:04.408481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:58:04.408481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 641/1265] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:04.414878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.528287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.551292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130104-159b-4339-91c7-3e2807da6c9d None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.598740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.599389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.599668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.599942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.600592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:04.600894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.602268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.619992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 625/1266] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:04.629010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.744745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb324e26-1768-4db0-9627-452c59089b98 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.766750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:04.781765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.782408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.782688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.782960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] PATH_INFO: `/v3/users/placement` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.791196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.792651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:04.805762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:04.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:58:04.808471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 642/1267] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:04.813991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:04.934238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d151cef-4b8f-4fa6-816b-c3ef4a865579 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:04.976987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:04.977604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:04.977884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:04.978155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:04.978893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:04.979231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:04.981498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:04.992016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.002593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 626/1268] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Aug 29 18:58:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:05.008520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.134553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947301a-0cfa-4f42-9c03-cd63eb614978 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.173210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.173973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.174258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.174620 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.188357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.190004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:05.207308 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:05.215582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:05.216545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:05.216545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:05.216545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:58:05.216545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 643/1269] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:05.217565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.323436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b39b07-94c0-4467-aa92-c940e73b5abb None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.361938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.362560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.362844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.363415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.363850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:05.364096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.365492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.378398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 627/1270] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:05.384556 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.421050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.494964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac683a-3f92-410d-87d9-21700246af62 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.532231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.532918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.533241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.533560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.541352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.542809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:05.554941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:05.555937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:58:05.556997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 644/1271] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:05.562304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.635164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.691214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cd728-3c1a-414d-9594-dd5d4394a7d3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.728903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ae3dd-d8de-4c77-940c-456a3d71ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.729513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae3dd-d8de-4c77-940c-456a3d71ad0d None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.729789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae3dd-d8de-4c77-940c-456a3d71ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.730061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae3dd-d8de-4c77-940c-456a3d71ad0d None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.730758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ae3dd-d8de-4c77-940c-456a3d71ad0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:05.731059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ae3dd-d8de-4c77-940c-456a3d71ad0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.732515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ae3dd-d8de-4c77-940c-456a3d71ad0d None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.744852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 628/1272] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:05.751638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:05.854536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:05.863641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4859-22ad-461e-9115-4dd62585f2a4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:05.900109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9895c-3ac1-4320-be94-557e02ae4079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:05.900677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9895c-3ac1-4320-be94-557e02ae4079 None admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:05.900955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9895c-3ac1-4320-be94-557e02ae4079 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:05.901228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9895c-3ac1-4320-be94-557e02ae4079 None admin] PATH_INFO: `/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/005b6400ae0c40bd8da5b9ddcc96770b/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:05.903140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9895c-3ac1-4320-be94-557e02ae4079 None admin] RBAC: Authorizing `identity:create_grant(project_id=91f1720f91bc43f594467ca8b5fe0c35, user_id=005b6400ae0c40bd8da5b9ddcc96770b, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:05.904665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9895c-3ac1-4320-be94-557e02ae4079 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:05.918216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 645/1273] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Aug 29 18:58:05 2022] PUT /identity/v3/projects/91f1720f91bc43f594467ca8b5fe0c35/users/005b6400ae0c40bd8da5b9ddcc96770b/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 18:58:07.887284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7414b-d70d-4668-9618-2893a1a6e3d3 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:07.887803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7414b-d70d-4668-9618-2893a1a6e3d3 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:07.887914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7414b-d70d-4668-9618-2893a1a6e3d3 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:07.889710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 629/1274] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:58:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:07.895029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977223f-101e-4ce2-b114-bd9c3ae7a2af None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:07.895586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977223f-101e-4ce2-b114-bd9c3ae7a2af None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:07.895586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977223f-101e-4ce2-b114-bd9c3ae7a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:07.915639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d977223f-101e-4ce2-b114-bd9c3ae7a2af None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:07.916875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d977223f-101e-4ce2-b114-bd9c3ae7a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:07.991919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 646/1275] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:58:08.283756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7414b-d70d-4668-9618-2893a1a6e3d3 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.399309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7414b-d70d-4668-9618-2893a1a6e3d3 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:08.417640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:08.439572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:08.440396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:08.440851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:08.441115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:08.455610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 18:58:08.462288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:08.463634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:08.464638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 630/1276] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:08.471861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977223f-101e-4ce2-b114-bd9c3ae7a2af None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.562929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:08.583659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977223f-101e-4ce2-b114-bd9c3ae7a2af None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:08.619736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:08.620351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:08.620633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:08.620904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:08.621575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:08.621880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:08.623283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:08.632635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 647/1277] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:08.638329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.753733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb36d1-9a27-4d40-8c3a-d208956e145b None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:08.798371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:08.799420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:08.803090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:08.803090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:08.813090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:08.814641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:08.830129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:08.831268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:08.832370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:08.832370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:08.832370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:08.832370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 631/1278] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:08.839336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:08.951434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ffeb4-295d-4ca4-83e1-fd08849b3310 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.004055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.004958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.005385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.005789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.006777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:09.007270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:09.009464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:09.028350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 648/1279] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:09.034906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.067775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.141661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19a90e-e163-4434-9164-b369a3838b61 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.181171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.181965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.182294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.182675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] PATH_INFO: `/v3/users/placement` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.195508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:09.196965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 18:58:09.209789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:09.211019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:09.212393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 632/1280] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:09.217040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.222500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.331822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e958b-abf3-4fa0-bb82-159ab3623bb1 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.372871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.373553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.373828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.374102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.374845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:09.375186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:09.376650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:09.396991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 649/1281] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:09.403091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.506464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.517384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ad38a-8e2a-4fb7-97f1-e41f1c86dabc None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.552676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.553282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.553558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.553826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.631773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:09.633136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:09.729478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.741414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 633/1282] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 18:58:09.747422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:09.814893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:09.857497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dd6b2-30d2-4bb3-b648-36e0edb3eec4 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:09.897486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:09.898041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:09.898321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:09.898590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:09.909449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:09.910920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 18:58:09.927917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:09.929191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 18:58:09.930102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 18:58:09.930102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 18:58:09.930102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:58:09.930102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 650/1283] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:09.939602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.046966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562b0ee-d195-429b-8823-9dab648fd8ce None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.085246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.085863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.086145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.086416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.087062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:10.087393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.089350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.112301 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 634/1284] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Aug 29 18:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.118172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.229043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c97c7f-f522-4a0c-b901-ae43de5d37d3 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.243778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.272378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.273010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.273287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.273558 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.281383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.282842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 18:58:10.294506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:10.296266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: ERROR keystone.server.flask.application Aug 29 18:58:10.297204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 651/1285] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 18:58:10.305074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.358496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:10.413992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b88d61-425a-4b17-b4d7-f3633d090795 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.452536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12809553-fe68-43b6-95ca-26a0c5581760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.453144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809553-fe68-43b6-95ca-26a0c5581760 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.453422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809553-fe68-43b6-95ca-26a0c5581760 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.453694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809553-fe68-43b6-95ca-26a0c5581760 None admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.454408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12809553-fe68-43b6-95ca-26a0c5581760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:10.454708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12809553-fe68-43b6-95ca-26a0c5581760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.456420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12809553-fe68-43b6-95ca-26a0c5581760 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.470688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 635/1286] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:10.476877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:10.580949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fea43-cde5-4857-bd6b-566108ca26d7 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:10.616295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e70d2c-b37d-4a37-891d-6f59320adb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:10.616907 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70d2c-b37d-4a37-891d-6f59320adb30 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:10.617187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70d2c-b37d-4a37-891d-6f59320adb30 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:10.617459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70d2c-b37d-4a37-891d-6f59320adb30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:10.618184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70d2c-b37d-4a37-891d-6f59320adb30 None admin] RBAC: Adding query filter params (role.id=89d008d3858c4e1b9ca6c28060b84405, scope.project.id=91f1720f91bc43f594467ca8b5fe0c35, user.id=005b6400ae0c40bd8da5b9ddcc96770b) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:10.618500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70d2c-b37d-4a37-891d-6f59320adb30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:10.619957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70d2c-b37d-4a37-891d-6f59320adb30 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:10.626145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 652/1287] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Aug 29 18:58:10 2022] GET /identity/v3/role_assignments?user.id=005b6400ae0c40bd8da5b9ddcc96770b&scope.project.id=91f1720f91bc43f594467ca8b5fe0c35&role.id=89d008d3858c4e1b9ca6c28060b84405 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:12.640351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ef63e-cc41-496b-adff-7a2b35bf8a9d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.640990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ef63e-cc41-496b-adff-7a2b35bf8a9d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.640990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ef63e-cc41-496b-adff-7a2b35bf8a9d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.642240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 636/1288] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:12.648134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0799-c0b8-4908-a608-5ca95fb370ba None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:12.648616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0799-c0b8-4908-a608-5ca95fb370ba None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:12.648736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0799-c0b8-4908-a608-5ca95fb370ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:12.667693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e06c0799-c0b8-4908-a608-5ca95fb370ba None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:12.668897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e06c0799-c0b8-4908-a608-5ca95fb370ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:12.701534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:12.754510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 653/1289] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:58:13.021146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ef63e-cc41-496b-adff-7a2b35bf8a9d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.133625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ef63e-cc41-496b-adff-7a2b35bf8a9d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.145894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:13.175681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.176611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.177017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.177537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.178342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.180066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:13.192399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:13.193621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:13.194517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 637/1290] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:13.202139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c0799-c0b8-4908-a608-5ca95fb370ba None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.322699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c0799-c0b8-4908-a608-5ca95fb370ba None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.357759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9e092-ac7e-43a7-afa7-5626ba7528fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.358946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9e092-ac7e-43a7-afa7-5626ba7528fb None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.359254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9e092-ac7e-43a7-afa7-5626ba7528fb None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.359526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9e092-ac7e-43a7-afa7-5626ba7528fb None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.360160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9e092-ac7e-43a7-afa7-5626ba7528fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:13.360460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9e092-ac7e-43a7-afa7-5626ba7528fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.361829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9e092-ac7e-43a7-afa7-5626ba7528fb None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.366928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 654/1291] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 18:58:13.372656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:13.477145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25efd-7793-45dc-970b-c0340d6aaeb6 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:13.512508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b3e0d-005c-4b08-a9a1-b7bec8cfe1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:13.513120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b3e0d-005c-4b08-a9a1-b7bec8cfe1a5 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:13.513402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b3e0d-005c-4b08-a9a1-b7bec8cfe1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:13.513676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b3e0d-005c-4b08-a9a1-b7bec8cfe1a5 None admin] PATH_INFO: `/v3/services` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:13.514359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b3e0d-005c-4b08-a9a1-b7bec8cfe1a5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:13.514897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b3e0d-005c-4b08-a9a1-b7bec8cfe1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:13.516303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b3e0d-005c-4b08-a9a1-b7bec8cfe1a5 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:13.521076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 638/1292] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Aug 29 18:58:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:15.446761 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dadf93-d4fb-45d3-b93b-defcfcbeb9b4 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:15.447487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dadf93-d4fb-45d3-b93b-defcfcbeb9b4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:15.447487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dadf93-d4fb-45d3-b93b-defcfcbeb9b4 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:15.448674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 655/1293] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:15.454182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28d2fa-c33f-4d79-9d2e-ef392f952949 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:15.454656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28d2fa-c33f-4d79-9d2e-ef392f952949 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:15.454751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28d2fa-c33f-4d79-9d2e-ef392f952949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:15.473143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-9c28d2fa-c33f-4d79-9d2e-ef392f952949 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:15.474347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9c28d2fa-c33f-4d79-9d2e-ef392f952949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:15.507086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:15.546350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 639/1294] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 18:58:15.823252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dadf93-d4fb-45d3-b93b-defcfcbeb9b4 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:15.857453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:15.942639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dadf93-d4fb-45d3-b93b-defcfcbeb9b4 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:15.979503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce320c2-0db0-495a-89d7-dce2d776fa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:15.980112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce320c2-0db0-495a-89d7-dce2d776fa85 None admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:15.980391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce320c2-0db0-495a-89d7-dce2d776fa85 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:15.980664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce320c2-0db0-495a-89d7-dce2d776fa85 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:15.981316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce320c2-0db0-495a-89d7-dce2d776fa85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:15.982684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce320c2-0db0-495a-89d7-dce2d776fa85 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:15.997631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 656/1295] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Aug 29 18:58:15 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:17.995917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1daab-5b1e-4f73-a15b-234203b57cb0 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:17.996450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1daab-5b1e-4f73-a15b-234203b57cb0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:17.996450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1daab-5b1e-4f73-a15b-234203b57cb0 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:17.997819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 640/1296] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:18.003388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f0a7-49ad-4ab3-9b78-9a8d1064d3ab None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.003836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f0a7-49ad-4ab3-9b78-9a8d1064d3ab None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.003949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f0a7-49ad-4ab3-9b78-9a8d1064d3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.022537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cda1f0a7-49ad-4ab3-9b78-9a8d1064d3ab None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:18.023769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cda1f0a7-49ad-4ab3-9b78-9a8d1064d3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:18.104609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 657/1297] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:58:18.388573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1daab-5b1e-4f73-a15b-234203b57cb0 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.492454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1daab-5b1e-4f73-a15b-234203b57cb0 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.528113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.528750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.529030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.529303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] PATH_INFO: `/v3/services/placement` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.529890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.531427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:18.540889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:18.542694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:18.543778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 641/1298] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:18.548122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f0a7-49ad-4ab3-9b78-9a8d1064d3ab None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.567664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.651916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f0a7-49ad-4ab3-9b78-9a8d1064d3ab None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.686973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.687713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.687992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.688263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.688895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] RBAC: Adding query filter params (name=placement) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:18.689199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.690574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.696065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 658/1299] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.702182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.740033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.822407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e40f7-2504-41f9-a7e1-b6be2e57e623 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:18.861544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc747d5-a41a-4512-b53b-f0cd571df829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:18.862158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc747d5-a41a-4512-b53b-f0cd571df829 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:18.862436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc747d5-a41a-4512-b53b-f0cd571df829 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:18.862706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc747d5-a41a-4512-b53b-f0cd571df829 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:18.863574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc747d5-a41a-4512-b53b-f0cd571df829 None admin] RBAC: Adding query filter params (interface=public, service_id=3b57c3e03401432c8cc71799da8bd8f4, region_id=RegionOne) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:18.863880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc747d5-a41a-4512-b53b-f0cd571df829 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:18.865242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc747d5-a41a-4512-b53b-f0cd571df829 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:18.870764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 642/1300] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/endpoints?service_id=3b57c3e03401432c8cc71799da8bd8f4&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:18.877243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:18.975846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:18.995787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419ff6f-60a6-4d2c-a896-e3898841d001 None admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:19.034064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4006b-5912-4b31-804f-d95ea7e284fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:19.034816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4006b-5912-4b31-804f-d95ea7e284fa None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:19.034973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4006b-5912-4b31-804f-d95ea7e284fa None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:19.035278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4006b-5912-4b31-804f-d95ea7e284fa None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:19.035889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4006b-5912-4b31-804f-d95ea7e284fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:19.037249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4006b-5912-4b31-804f-d95ea7e284fa None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:19.044210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 659/1301] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Aug 29 18:58:18 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 18:58:21.053991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9cef-c731-4412-99ae-096c242937ab None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.054560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9cef-c731-4412-99ae-096c242937ab None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.054560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9cef-c731-4412-99ae-096c242937ab None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.055941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 643/1302] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:58:21.061302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d46b3-4cbc-448f-92ac-04048ca00e26 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.061921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d46b3-4cbc-448f-92ac-04048ca00e26 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.062200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d46b3-4cbc-448f-92ac-04048ca00e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.080882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1f7d46b3-4cbc-448f-92ac-04048ca00e26 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:58:21.082091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1f7d46b3-4cbc-448f-92ac-04048ca00e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:58:21.183561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 660/1303] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:58:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 18:58:21.476343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9cef-c731-4412-99ae-096c242937ab None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:21.580284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:21.588189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9cef-c731-4412-99ae-096c242937ab None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:21.628787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:21.629434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.629717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.629993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.630589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:21.632370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 18:58:21.642672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 18:58:21.644018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application with Lock( Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self._enter() Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application return self.creator() Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application created_value = creator( Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: ERROR keystone.server.flask.application Aug 29 18:58:21.644957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 644/1304] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Aug 29 18:58:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 18:58:21.650347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d46b3-4cbc-448f-92ac-04048ca00e26 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:21.729579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:21.753707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d46b3-4cbc-448f-92ac-04048ca00e26 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:21.789523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc6116-507b-4c3b-864c-5583c25671a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:21.790135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc6116-507b-4c3b-864c-5583c25671a3 None admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.790415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc6116-507b-4c3b-864c-5583c25671a3 None admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.790689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc6116-507b-4c3b-864c-5583c25671a3 None admin] PATH_INFO: `/v3/services` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.791343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc6116-507b-4c3b-864c-5583c25671a3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 18:58:21.791646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc6116-507b-4c3b-864c-5583c25671a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:21.793018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc6116-507b-4c3b-864c-5583c25671a3 None admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:21.798538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 661/1305] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Aug 29 18:58:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 18:58:21.804659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:58:21.914058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537bd6d-d61a-42ae-ab67-47cf67e0aae3 None admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:58:21.950683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca863988-1a82-4426-80b0-97966bbaa259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:58:21.951326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca863988-1a82-4426-80b0-97966bbaa259 None admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:58:21.951609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca863988-1a82-4426-80b0-97966bbaa259 None admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:58:21.951883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca863988-1a82-4426-80b0-97966bbaa259 None admin] PATH_INFO: `/v3/endpoints` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:58:21.953190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca863988-1a82-4426-80b0-97966bbaa259 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:58:21.954562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca863988-1a82-4426-80b0-97966bbaa259 None admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:58:21.980200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 645/1306] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 18:58:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 18:58:31.795749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=30 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:58:31.974543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:29.722337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127a67-17bc-49e6-b3f5-168dea7e134c None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.723346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127a67-17bc-49e6-b3f5-168dea7e134c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.723837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127a67-17bc-49e6-b3f5-168dea7e134c None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.731524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cb98-e016-45f6-9555-ec6e3d7fa3a4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:29.732375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cb98-e016-45f6-9555-ec6e3d7fa3a4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:29.732835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77cb98-e016-45f6-9555-ec6e3d7fa3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:29.736773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 662/1307] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:29 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:29.933066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-4f77cb98-e016-45f6-9555-ec6e3d7fa3a4 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:29.934738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4f77cb98-e016-45f6-9555-ec6e3d7fa3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:30.082724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 646/1308] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:59:30.387415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e64e8-8e3f-4473-8c63-b348c37d4990 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.387703 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e64e8-8e3f-4473-8c63-b348c37d4990 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.388172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e64e8-8e3f-4473-8c63-b348c37d4990 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.389693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 663/1309] 10.222.0.94 () {54 vars in 965 bytes} [Mon Aug 29 18:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:30.396819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaf5ca-d730-4337-b67a-9f9c92af7214 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.397112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaf5ca-d730-4337-b67a-9f9c92af7214 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.397401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaf5ca-d730-4337-b67a-9f9c92af7214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.486039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-84eaf5ca-d730-4337-b67a-9f9c92af7214 None None] MFA Rules not processed for user `70c6682b92694484b2fa2710314f547f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:30.487443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-84eaf5ca-d730-4337-b67a-9f9c92af7214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:30.525941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:30.601487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 647/1310] 10.222.0.94 () {58 vars in 1062 bytes} [Mon Aug 29 18:59:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:59:30.611399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e64e8-8e3f-4473-8c63-b348c37d4990 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:30.613955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-331e64e8-8e3f-4473-8c63-b348c37d4990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:30.743609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e64e8-8e3f-4473-8c63-b348c37d4990 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:30.840457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b5fd9-dff4-4134-af66-42c7fd7d79e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:30.841088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b5fd9-dff4-4134-af66-42c7fd7d79e4 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:30.841372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b5fd9-dff4-4134-af66-42c7fd7d79e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:30.841644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b5fd9-dff4-4134-af66-42c7fd7d79e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:30.843483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ca8b5fd9-dff4-4134-af66-42c7fd7d79e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:30.911982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:30.980718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b5fd9-dff4-4134-af66-42c7fd7d79e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:30.982444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b5fd9-dff4-4134-af66-42c7fd7d79e4 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:31.005539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 664/1311] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 18:59:33.600632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744eab-b5a2-46d0-8734-983c3599ea58 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:33.601200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744eab-b5a2-46d0-8734-983c3599ea58 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:33.601200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744eab-b5a2-46d0-8734-983c3599ea58 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:33.602528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 648/1312] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:33.608056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9fcd5-aecf-4a9c-8f1f-f7b3e4e5919d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:33.608533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9fcd5-aecf-4a9c-8f1f-f7b3e4e5919d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:33.608625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9fcd5-aecf-4a9c-8f1f-f7b3e4e5919d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:33.683803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7cd9fcd5-aecf-4a9c-8f1f-f7b3e4e5919d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:33.685051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7cd9fcd5-aecf-4a9c-8f1f-f7b3e4e5919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:33.786546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 665/1313] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:34.074597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab744eab-b5a2-46d0-8734-983c3599ea58 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:34.076182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ab744eab-b5a2-46d0-8734-983c3599ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:34.194457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab744eab-b5a2-46d0-8734-983c3599ea58 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:34.276234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:34.279605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355dc05-edde-4116-8480-52a9a28fbda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:34.280464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355dc05-edde-4116-8480-52a9a28fbda9 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:34.280910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355dc05-edde-4116-8480-52a9a28fbda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:34.281353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355dc05-edde-4116-8480-52a9a28fbda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:34.283835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4355dc05-edde-4116-8480-52a9a28fbda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:34.401201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355dc05-edde-4116-8480-52a9a28fbda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:34.402832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355dc05-edde-4116-8480-52a9a28fbda9 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:34.425610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 649/1314] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:36.730495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cbdda-46e3-4836-8a15-58b4ea2e3b3f None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:36.731060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cbdda-46e3-4836-8a15-58b4ea2e3b3f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:36.731060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cbdda-46e3-4836-8a15-58b4ea2e3b3f None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:36.732402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 666/1315] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:36.737664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d593-f316-4a77-bc94-3f499ce1dd83 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:36.738101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d593-f316-4a77-bc94-3f499ce1dd83 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:36.738198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d593-f316-4a77-bc94-3f499ce1dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:36.807223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7b59d593-f316-4a77-bc94-3f499ce1dd83 None None] MFA Rules not processed for user `81c4f0f0b041419391d31004afc75480`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:36.808441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7b59d593-f316-4a77-bc94-3f499ce1dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:36.861514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:36.906818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 650/1316] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:59:37.192760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cbdda-46e3-4836-8a15-58b4ea2e3b3f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:37.194460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-654cbdda-46e3-4836-8a15-58b4ea2e3b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:37.204292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:37.299242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cbdda-46e3-4836-8a15-58b4ea2e3b3f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:37.374344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2df83-d702-4d9c-bca1-8c535c7c5fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:37.374954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2df83-d702-4d9c-bca1-8c535c7c5fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:37.375261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2df83-d702-4d9c-bca1-8c535c7c5fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:37.375536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2df83-d702-4d9c-bca1-8c535c7c5fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:37.377687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8ec2df83-d702-4d9c-bca1-8c535c7c5fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:37.443501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:37.493014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2df83-d702-4d9c-bca1-8c535c7c5fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:37.494517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2df83-d702-4d9c-bca1-8c535c7c5fb2 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:37.515513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 667/1317] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:40.011681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06aa25-5e1c-47e6-9fe2-0e1643da86ae None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.012215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06aa25-5e1c-47e6-9fe2-0e1643da86ae None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.012215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06aa25-5e1c-47e6-9fe2-0e1643da86ae None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.013580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 651/1318] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:40.019236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969a23-472c-4498-bb11-38aa9cb8df1f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.019661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969a23-472c-4498-bb11-38aa9cb8df1f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.019826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969a23-472c-4498-bb11-38aa9cb8df1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.090811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-28969a23-472c-4498-bb11-38aa9cb8df1f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:40.091969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-28969a23-472c-4498-bb11-38aa9cb8df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:40.199831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 668/1319] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 18:59:40.497082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ca930-1817-464b-a891-d995a7f219df None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.497366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ca930-1817-464b-a891-d995a7f219df None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.497639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ca930-1817-464b-a891-d995a7f219df None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.499057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 652/1320] 10.222.0.94 () {54 vars in 965 bytes} [Mon Aug 29 18:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:40.505233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93342f47-837b-4060-a4bd-a44d7e412ac3 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.505518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93342f47-837b-4060-a4bd-a44d7e412ac3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.505830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93342f47-837b-4060-a4bd-a44d7e412ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.528348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:40.584699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-93342f47-837b-4060-a4bd-a44d7e412ac3 None None] MFA Rules not processed for user `70c6682b92694484b2fa2710314f547f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:40.585934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-93342f47-837b-4060-a4bd-a44d7e412ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:40.665544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 669/1321] 10.222.0.94 () {58 vars in 1062 bytes} [Mon Aug 29 18:59:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:40.674949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ca930-1817-464b-a891-d995a7f219df None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:40.676539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5c8ca930-1817-464b-a891-d995a7f219df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:40.801091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ca930-1817-464b-a891-d995a7f219df None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:40.896032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ff293-8068-4c7d-965c-360353e53a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:40.896641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff293-8068-4c7d-965c-360353e53a03 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:40.896919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff293-8068-4c7d-965c-360353e53a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:40.897223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff293-8068-4c7d-965c-360353e53a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:40.898864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f56ff293-8068-4c7d-965c-360353e53a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:40.906811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:41.012939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ff293-8068-4c7d-965c-360353e53a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:41.014527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ff293-8068-4c7d-965c-360353e53a03 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:41.034000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 653/1322] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 18:59:44.295261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8253685-5a7a-4e82-81bc-a2efdee9cfe9 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.295768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8253685-5a7a-4e82-81bc-a2efdee9cfe9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.295768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8253685-5a7a-4e82-81bc-a2efdee9cfe9 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.297120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 670/1323] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 18:59:44.303654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e567a-aedf-40cd-9621-3e8827cd1fa3 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.304110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e567a-aedf-40cd-9621-3e8827cd1fa3 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.304222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e567a-aedf-40cd-9621-3e8827cd1fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.375671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-788e567a-aedf-40cd-9621-3e8827cd1fa3 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:44.376933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-788e567a-aedf-40cd-9621-3e8827cd1fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:44.440333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:44.483612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 654/1324] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:44.775613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8253685-5a7a-4e82-81bc-a2efdee9cfe9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:44.777219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d8253685-5a7a-4e82-81bc-a2efdee9cfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:44.905722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8253685-5a7a-4e82-81bc-a2efdee9cfe9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:44.919670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:44.985744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae745335-1b64-47c6-a95b-d8e3ad64fa5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:44.986480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae745335-1b64-47c6-a95b-d8e3ad64fa5a service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:44.986740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae745335-1b64-47c6-a95b-d8e3ad64fa5a service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:44.987014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae745335-1b64-47c6-a95b-d8e3ad64fa5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:44.988734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ae745335-1b64-47c6-a95b-d8e3ad64fa5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:45.110129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae745335-1b64-47c6-a95b-d8e3ad64fa5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:45.112359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae745335-1b64-47c6-a95b-d8e3ad64fa5a service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:45.134866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 671/1325] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:47.177987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23530bdb-e14b-404e-9a4e-a735a0b594f0 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.178530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23530bdb-e14b-404e-9a4e-a735a0b594f0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.178868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23530bdb-e14b-404e-9a4e-a735a0b594f0 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.181384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 655/1326] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:47.190288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c545-0635-4300-9de8-3972c4c3f6f8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.191052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c545-0635-4300-9de8-3972c4c3f6f8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.191052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c545-0635-4300-9de8-3972c4c3f6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.249109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:47.270033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-dbd5c545-0635-4300-9de8-3972c4c3f6f8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:47.271424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dbd5c545-0635-4300-9de8-3972c4c3f6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:47.387849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 672/1327] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:47.685093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23530bdb-e14b-404e-9a4e-a735a0b594f0 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:47.686920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-23530bdb-e14b-404e-9a4e-a735a0b594f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:47.812736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23530bdb-e14b-404e-9a4e-a735a0b594f0 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:47.913795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec805fa-a032-42ac-93fd-dc93acabf109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:47.914054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec805fa-a032-42ac-93fd-dc93acabf109 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:47.914175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec805fa-a032-42ac-93fd-dc93acabf109 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:47.914406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec805fa-a032-42ac-93fd-dc93acabf109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:47.916090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-eec805fa-a032-42ac-93fd-dc93acabf109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:47.934300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:48.051665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec805fa-a032-42ac-93fd-dc93acabf109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:48.053108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec805fa-a032-42ac-93fd-dc93acabf109 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:48.072683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 656/1328] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 18:59:50.110287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555d4d2-8c24-47b0-942f-9880ff00a7d6 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.110808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555d4d2-8c24-47b0-942f-9880ff00a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.110808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555d4d2-8c24-47b0-942f-9880ff00a7d6 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.113032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 673/1329] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:50.118855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bd302-adf9-4dee-8e1a-cbeb58be2ac4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.119420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bd302-adf9-4dee-8e1a-cbeb58be2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.119420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bd302-adf9-4dee-8e1a-cbeb58be2ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.197320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-941bd302-adf9-4dee-8e1a-cbeb58be2ac4 None None] MFA Rules not processed for user `81c4f0f0b041419391d31004afc75480`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:50.198545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-941bd302-adf9-4dee-8e1a-cbeb58be2ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:50.273398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:50.307367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 657/1330] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:50.599687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555d4d2-8c24-47b0-942f-9880ff00a7d6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:50.601333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8555d4d2-8c24-47b0-942f-9880ff00a7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:50.716977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555d4d2-8c24-47b0-942f-9880ff00a7d6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:50.737575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:50.793068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7719d-18cb-4ec6-b90b-178842fde4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:50.793596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7719d-18cb-4ec6-b90b-178842fde4af service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:50.793870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7719d-18cb-4ec6-b90b-178842fde4af service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:50.794137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7719d-18cb-4ec6-b90b-178842fde4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:50.795756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2de7719d-18cb-4ec6-b90b-178842fde4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:50.910171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7719d-18cb-4ec6-b90b-178842fde4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:50.911668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7719d-18cb-4ec6-b90b-178842fde4af service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:50.930838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 674/1331] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:53.319544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74fd8-02fe-4a0f-9ee7-e39042635d07 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.320133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74fd8-02fe-4a0f-9ee7-e39042635d07 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.320235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74fd8-02fe-4a0f-9ee7-e39042635d07 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.321735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 658/1332] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:53.327151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f542b-8bde-465a-9696-942bbf34dd98 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.327601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f542b-8bde-465a-9696-942bbf34dd98 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.327771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f542b-8bde-465a-9696-942bbf34dd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.397536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:53.400846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-473f542b-8bde-465a-9696-942bbf34dd98 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:53.402395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-473f542b-8bde-465a-9696-942bbf34dd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:53.513273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 675/1333] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:53.793756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f74fd8-02fe-4a0f-9ee7-e39042635d07 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:53.795349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-79f74fd8-02fe-4a0f-9ee7-e39042635d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:53.901822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f74fd8-02fe-4a0f-9ee7-e39042635d07 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:53.976067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edb453-442c-4dc6-b685-af01800cd2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:53.976676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edb453-442c-4dc6-b685-af01800cd2fa service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:53.976956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edb453-442c-4dc6-b685-af01800cd2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:53.977230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edb453-442c-4dc6-b685-af01800cd2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:53.978841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-07edb453-442c-4dc6-b685-af01800cd2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:54.006220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:54.100734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edb453-442c-4dc6-b685-af01800cd2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:54.102192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edb453-442c-4dc6-b685-af01800cd2fa service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:54.120903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 659/1334] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:56.098643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1f9c-b983-41a9-9193-5f0bfa62792a None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.099170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1f9c-b983-41a9-9193-5f0bfa62792a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.099170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1f9c-b983-41a9-9193-5f0bfa62792a None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.100511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 676/1335] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:56.105591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603724-288a-43dd-af90-bc439e2b754c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.106017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603724-288a-43dd-af90-bc439e2b754c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.106148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603724-288a-43dd-af90-bc439e2b754c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.176063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d2603724-288a-43dd-af90-bc439e2b754c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:56.177353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d2603724-288a-43dd-af90-bc439e2b754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:56.256945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:56.280141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 660/1336] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:56.577279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f1f9c-b983-41a9-9193-5f0bfa62792a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:56.578821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-fe8f1f9c-b983-41a9-9193-5f0bfa62792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:56.693538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f1f9c-b983-41a9-9193-5f0bfa62792a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 18:59:56.723633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:56.783345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3d8fe-eb06-427f-88ee-0784a3493743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 18:59:56.785283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d8fe-eb06-427f-88ee-0784a3493743 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:56.785563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d8fe-eb06-427f-88ee-0784a3493743 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:56.785828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d8fe-eb06-427f-88ee-0784a3493743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:56.787530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-66a3d8fe-eb06-427f-88ee-0784a3493743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:56.947396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3d8fe-eb06-427f-88ee-0784a3493743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 18:59:56.949306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3d8fe-eb06-427f-88ee-0784a3493743 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 18:59:56.968722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 677/1337] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 18:59:59.372344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc238-d3fb-43ed-b3b0-833cbf78f700 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:59.372916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc238-d3fb-43ed-b3b0-833cbf78f700 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:59.373018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc238-d3fb-43ed-b3b0-833cbf78f700 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:59.374552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 661/1338] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 18:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 18:59:59.380891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4741b-d444-47a3-bf75-0bd5df94530e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 18:59:59.381354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4741b-d444-47a3-bf75-0bd5df94530e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 18:59:59.381477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4741b-d444-47a3-bf75-0bd5df94530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 18:59:59.448551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-16a4741b-d444-47a3-bf75-0bd5df94530e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 18:59:59.449679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-16a4741b-d444-47a3-bf75-0bd5df94530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 18:59:59.458314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 18:59:59.560106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 678/1339] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 18:59:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 18:59:59.858728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bc238-d3fb-43ed-b3b0-833cbf78f700 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 18:59:59.860332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-613bc238-d3fb-43ed-b3b0-833cbf78f700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:00.000239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bc238-d3fb-43ed-b3b0-833cbf78f700 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:00.092905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18350baa-364e-4ca8-8ad3-62b69a16dc1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:00.093467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18350baa-364e-4ca8-8ad3-62b69a16dc1f service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:00.093744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18350baa-364e-4ca8-8ad3-62b69a16dc1f service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:00.094014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18350baa-364e-4ca8-8ad3-62b69a16dc1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:00.095642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-18350baa-364e-4ca8-8ad3-62b69a16dc1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:00.135903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:00.214017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18350baa-364e-4ca8-8ad3-62b69a16dc1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:00.215486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18350baa-364e-4ca8-8ad3-62b69a16dc1f service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:00.235392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 662/1340] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 18:59:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:04.335130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ad47-7dca-4b50-94b7-7c55e8eb1269 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:04.335563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ad47-7dca-4b50-94b7-7c55e8eb1269 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:04.335839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ad47-7dca-4b50-94b7-7c55e8eb1269 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:04.337225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 679/1341] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:04.342961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e7cb-f396-4611-b778-6e9264e593a2 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:04.343843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e7cb-f396-4611-b778-6e9264e593a2 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:04.343843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e7cb-f396-4611-b778-6e9264e593a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:04.416631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a2b0e7cb-f396-4611-b778-6e9264e593a2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:04.417849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a2b0e7cb-f396-4611-b778-6e9264e593a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:04.508177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:04.521226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 663/1342] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:04.817950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ad47-7dca-4b50-94b7-7c55e8eb1269 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:04.819536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ddc8ad47-7dca-4b50-94b7-7c55e8eb1269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:04.934513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ad47-7dca-4b50-94b7-7c55e8eb1269 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:04.975115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:05.014425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6e174-bc04-471d-aaf9-c16625e5e8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:05.015040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6e174-bc04-471d-aaf9-c16625e5e8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:05.015349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6e174-bc04-471d-aaf9-c16625e5e8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:05.015621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6e174-bc04-471d-aaf9-c16625e5e8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:05.017242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f3a6e174-bc04-471d-aaf9-c16625e5e8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:05.135039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a6e174-bc04-471d-aaf9-c16625e5e8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:05.136462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a6e174-bc04-471d-aaf9-c16625e5e8a6 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:05.157694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 680/1343] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:07.826925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef205-f361-4119-b5ae-e781fef30417 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:07.827575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef205-f361-4119-b5ae-e781fef30417 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:07.827673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef205-f361-4119-b5ae-e781fef30417 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:07.829415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 664/1344] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:07.836332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2078a3b-d085-4f91-b2c7-5078514b0b1f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:07.836332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2078a3b-d085-4f91-b2c7-5078514b0b1f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:07.836332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2078a3b-d085-4f91-b2c7-5078514b0b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:07.929791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d2078a3b-d085-4f91-b2c7-5078514b0b1f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:07.931601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d2078a3b-d085-4f91-b2c7-5078514b0b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:07.955543 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:08.058656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 681/1345] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:08.362976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef205-f361-4119-b5ae-e781fef30417 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:08.364577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cfbef205-f361-4119-b5ae-e781fef30417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:08.473914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef205-f361-4119-b5ae-e781fef30417 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:08.562151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98943b-4c60-49d6-91c5-ec0e52c60bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:08.562766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98943b-4c60-49d6-91c5-ec0e52c60bcf service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:08.563079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98943b-4c60-49d6-91c5-ec0e52c60bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:08.563347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98943b-4c60-49d6-91c5-ec0e52c60bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:08.564955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0b98943b-4c60-49d6-91c5-ec0e52c60bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:08.611888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:08.688596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98943b-4c60-49d6-91c5-ec0e52c60bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:08.690033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98943b-4c60-49d6-91c5-ec0e52c60bcf service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:08.710304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 665/1346] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:12.753624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819aad4-b409-4483-a57e-b27b12fbed6f None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:12.753624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819aad4-b409-4483-a57e-b27b12fbed6f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:12.754516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819aad4-b409-4483-a57e-b27b12fbed6f None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:12.755335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 682/1347] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:12.760957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae1e1-fbcb-489f-9401-829d57f7256b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:12.761431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae1e1-fbcb-489f-9401-829d57f7256b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:12.761526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae1e1-fbcb-489f-9401-829d57f7256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:12.843277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-23dae1e1-fbcb-489f-9401-829d57f7256b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:12.844527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-23dae1e1-fbcb-489f-9401-829d57f7256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:12.980642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 666/1348] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:12.983281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:13.283211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819aad4-b409-4483-a57e-b27b12fbed6f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:13.286149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4819aad4-b409-4483-a57e-b27b12fbed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:13.443326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819aad4-b409-4483-a57e-b27b12fbed6f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:13.509603 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:13.551771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ea00c-6bc3-4e31-8356-ed706b8d5d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:13.552537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea00c-6bc3-4e31-8356-ed706b8d5d5f service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:13.552880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea00c-6bc3-4e31-8356-ed706b8d5d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:13.553317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea00c-6bc3-4e31-8356-ed706b8d5d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:13.555278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-333ea00c-6bc3-4e31-8356-ed706b8d5d5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:13.722395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ea00c-6bc3-4e31-8356-ed706b8d5d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:13.724051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ea00c-6bc3-4e31-8356-ed706b8d5d5f service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:13.743676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 683/1349] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:00:18.995861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ddd1b-a9b6-4e22-ac66-c9d068d1336a None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:18.996383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ddd1b-a9b6-4e22-ac66-c9d068d1336a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:18.996479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ddd1b-a9b6-4e22-ac66-c9d068d1336a None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:18.997910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 667/1350] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:19.005711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af101b9-f3c2-42f3-a4cc-335d5b6bd697 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:19.005711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af101b9-f3c2-42f3-a4cc-335d5b6bd697 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:19.005711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af101b9-f3c2-42f3-a4cc-335d5b6bd697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:19.121299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0af101b9-f3c2-42f3-a4cc-335d5b6bd697 None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:19.122600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0af101b9-f3c2-42f3-a4cc-335d5b6bd697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:19.192214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:19.304067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 684/1351] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:00:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:23.777336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f0968-b3aa-4d40-82d8-7032c1ff24fc None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:23.777926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f0968-b3aa-4d40-82d8-7032c1ff24fc None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:23.777926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f0968-b3aa-4d40-82d8-7032c1ff24fc None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:23.779553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 668/1352] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:23.785999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18ab2-1f80-4b7f-8690-a7740d38a229 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:23.786564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18ab2-1f80-4b7f-8690-a7740d38a229 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:23.786564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18ab2-1f80-4b7f-8690-a7740d38a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:23.885784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8de18ab2-1f80-4b7f-8690-a7740d38a229 None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:23.887420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8de18ab2-1f80-4b7f-8690-a7740d38a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:23.978049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:24.026453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 685/1353] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:00:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:25.282285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac241e-f5d6-4a12-8303-2d40e131d6cf None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:25.282835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac241e-f5d6-4a12-8303-2d40e131d6cf None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:25.282835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac241e-f5d6-4a12-8303-2d40e131d6cf None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:25.284251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 669/1354] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:25.290850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ff856-45d9-498c-8002-b10527f15bf3 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:25.291407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ff856-45d9-498c-8002-b10527f15bf3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:25.291516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ff856-45d9-498c-8002-b10527f15bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:25.376575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d77ff856-45d9-498c-8002-b10527f15bf3 None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:25.377834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d77ff856-45d9-498c-8002-b10527f15bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:25.497070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 686/1355] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:00:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:28.949377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267cc8c-952f-4eae-9b62-2a5dfbc58cd0 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:28.949932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267cc8c-952f-4eae-9b62-2a5dfbc58cd0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:28.950031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267cc8c-952f-4eae-9b62-2a5dfbc58cd0 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:28.951365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 670/1356] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:00:28.962363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508b1ec-4afe-4446-8db4-1cb6e07c9c6f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:28.962363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508b1ec-4afe-4446-8db4-1cb6e07c9c6f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:28.962363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508b1ec-4afe-4446-8db4-1cb6e07c9c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:28.994833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:29.028848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-5508b1ec-4afe-4446-8db4-1cb6e07c9c6f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:29.030163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-5508b1ec-4afe-4446-8db4-1cb6e07c9c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:29.175095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 687/1357] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:29.490304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ce3f-dfbb-4412-b5da-ae34b586f8cd None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:29.490593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ce3f-dfbb-4412-b5da-ae34b586f8cd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:29.491224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ce3f-dfbb-4412-b5da-ae34b586f8cd None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:29.492576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 671/1358] 10.222.0.94 () {54 vars in 962 bytes} [Mon Aug 29 19:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:29.502690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5bae4-a28b-456b-b9ec-ada948a79334 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:29.503168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5bae4-a28b-456b-b9ec-ada948a79334 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:29.503631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5bae4-a28b-456b-b9ec-ada948a79334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:29.621229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b4d5bae4-a28b-456b-b9ec-ada948a79334 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:29.622605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b4d5bae4-a28b-456b-b9ec-ada948a79334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:29.682342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:29.757943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 688/1359] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Aug 29 19:00:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:29.765907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ce3f-dfbb-4412-b5da-ae34b586f8cd None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:29.767607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-55e5ce3f-dfbb-4412-b5da-ae34b586f8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:29.921815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ce3f-dfbb-4412-b5da-ae34b586f8cd None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:30.013186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:30.063686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f189f-b887-4b4b-9668-2e5318ab229d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:30.064303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f189f-b887-4b4b-9668-2e5318ab229d service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:30.064585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f189f-b887-4b4b-9668-2e5318ab229d service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:30.064915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f189f-b887-4b4b-9668-2e5318ab229d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:30.067385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5f8f189f-b887-4b4b-9668-2e5318ab229d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:30.165793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f189f-b887-4b4b-9668-2e5318ab229d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:30.167423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f189f-b887-4b4b-9668-2e5318ab229d service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:30.191924 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 672/1360] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:32.362856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81472c-01cd-4847-804f-9d125be1d14c None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:32.363643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81472c-01cd-4847-804f-9d125be1d14c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:32.363643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81472c-01cd-4847-804f-9d125be1d14c None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:32.364954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 689/1361] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:00:32.370014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf112-1d39-492c-846c-7d8766547921 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:32.370425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf112-1d39-492c-846c-7d8766547921 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:32.370733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf112-1d39-492c-846c-7d8766547921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:32.446957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-979bf112-1d39-492c-846c-7d8766547921 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:32.448213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-979bf112-1d39-492c-846c-7d8766547921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:32.495056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:32.567136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 673/1362] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:32.857173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81472c-01cd-4847-804f-9d125be1d14c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:32.858818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-fc81472c-01cd-4847-804f-9d125be1d14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:33.043499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81472c-01cd-4847-804f-9d125be1d14c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:33.054523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:33.168127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdec2e-2552-433a-bf56-1a0f8ba50878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:33.168742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdec2e-2552-433a-bf56-1a0f8ba50878 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:33.169017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdec2e-2552-433a-bf56-1a0f8ba50878 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:33.169393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdec2e-2552-433a-bf56-1a0f8ba50878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:33.170862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d2fdec2e-2552-433a-bf56-1a0f8ba50878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:33.267643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdec2e-2552-433a-bf56-1a0f8ba50878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:33.269237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdec2e-2552-433a-bf56-1a0f8ba50878 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:33.290115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 690/1363] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:35.545232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d086763-050b-4741-88aa-39e0dff5e058 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:35.546084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d086763-050b-4741-88aa-39e0dff5e058 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:35.546371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d086763-050b-4741-88aa-39e0dff5e058 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:35.547813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 674/1364] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:35.554728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca1644-f3d0-499f-9bcc-964795949a03 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:35.555256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca1644-f3d0-499f-9bcc-964795949a03 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:35.555347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca1644-f3d0-499f-9bcc-964795949a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:35.575033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:35.639852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-24ca1644-f3d0-499f-9bcc-964795949a03 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:35.641082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-24ca1644-f3d0-499f-9bcc-964795949a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:35.763573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 691/1365] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:36.067574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501bec6-1c10-4710-8c1c-fd578d779927 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:36.068181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501bec6-1c10-4710-8c1c-fd578d779927 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:36.068596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501bec6-1c10-4710-8c1c-fd578d779927 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:36.070689 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 675/1366] 10.222.0.94 () {54 vars in 962 bytes} [Mon Aug 29 19:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:36.077325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821e6e-eb4f-4ad5-a43c-82744bd6f18e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:36.077850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821e6e-eb4f-4ad5-a43c-82744bd6f18e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:36.078196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821e6e-eb4f-4ad5-a43c-82744bd6f18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:36.161498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ea821e6e-eb4f-4ad5-a43c-82744bd6f18e None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:36.162717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ea821e6e-eb4f-4ad5-a43c-82744bd6f18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:36.203841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:36.312323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 692/1367] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Aug 29 19:00:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:36.322189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501bec6-1c10-4710-8c1c-fd578d779927 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:36.324727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0501bec6-1c10-4710-8c1c-fd578d779927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:36.485456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501bec6-1c10-4710-8c1c-fd578d779927 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:36.508800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:36.620369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff371c7-dce4-46a0-a8bc-738a75e47517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:36.621055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff371c7-dce4-46a0-a8bc-738a75e47517 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:36.621286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff371c7-dce4-46a0-a8bc-738a75e47517 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:36.621514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff371c7-dce4-46a0-a8bc-738a75e47517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:36.623389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fff371c7-dce4-46a0-a8bc-738a75e47517 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:36.725785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff371c7-dce4-46a0-a8bc-738a75e47517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:36.727643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff371c7-dce4-46a0-a8bc-738a75e47517 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:36.747439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 676/1368] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:37.412277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9cda4-a793-4ae0-8641-409c843b3087 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:37.412813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9cda4-a793-4ae0-8641-409c843b3087 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:37.412813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9cda4-a793-4ae0-8641-409c843b3087 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:37.414325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 693/1369] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:37.419427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f058aa7-640c-4051-9794-c02fedb8e275 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:37.419653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f058aa7-640c-4051-9794-c02fedb8e275 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:37.419926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f058aa7-640c-4051-9794-c02fedb8e275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:37.455236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:37.497303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5f058aa7-640c-4051-9794-c02fedb8e275 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:37.498513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5f058aa7-640c-4051-9794-c02fedb8e275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:37.606416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 677/1370] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:38.174705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3945727-f952-4c21-a981-f5f683886f1c None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:38.174986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3945727-f952-4c21-a981-f5f683886f1c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:38.175292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3945727-f952-4c21-a981-f5f683886f1c None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:38.177816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 694/1371] 10.222.0.94 () {54 vars in 964 bytes} [Mon Aug 29 19:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:38.182749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7870da9-13bd-4b81-b4f5-dfdd9c6f596e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:38.183243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7870da9-13bd-4b81-b4f5-dfdd9c6f596e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:38.183338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7870da9-13bd-4b81-b4f5-dfdd9c6f596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:38.264920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d7870da9-13bd-4b81-b4f5-dfdd9c6f596e None None] MFA Rules not processed for user `ee56cf453075431ab2791e15e803d51c`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:38.266173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d7870da9-13bd-4b81-b4f5-dfdd9c6f596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:38.319236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:38.353021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 678/1372] 10.222.0.94 () {58 vars in 1061 bytes} [Mon Aug 29 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:38.360485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3945727-f952-4c21-a981-f5f683886f1c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:38.362034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f3945727-f952-4c21-a981-f5f683886f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:38.474116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3945727-f952-4c21-a981-f5f683886f1c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:38.521172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:38.568886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e1bd6-b5f7-4c0d-bf25-5c4334c43589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:38.569504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e1bd6-b5f7-4c0d-bf25-5c4334c43589 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:38.569793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e1bd6-b5f7-4c0d-bf25-5c4334c43589 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:38.570106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e1bd6-b5f7-4c0d-bf25-5c4334c43589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:38.571704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-527e1bd6-b5f7-4c0d-bf25-5c4334c43589 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:38.765260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e1bd6-b5f7-4c0d-bf25-5c4334c43589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:38.766518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e1bd6-b5f7-4c0d-bf25-5c4334c43589 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:38.803096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 695/1373] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:38.808222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1333d-865c-4a3d-8787-1b2e64283bf0 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:38.808766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1333d-865c-4a3d-8787-1b2e64283bf0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:38.809170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1333d-865c-4a3d-8787-1b2e64283bf0 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:38.811226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 679/1374] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:38.816804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799202b9-d410-4078-801e-95ed5c8112f4 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:38.817115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799202b9-d410-4078-801e-95ed5c8112f4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:38.817464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799202b9-d410-4078-801e-95ed5c8112f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:38.917002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-799202b9-d410-4078-801e-95ed5c8112f4 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:38.918444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-799202b9-d410-4078-801e-95ed5c8112f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:38.944573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:39.026591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 696/1375] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:00:39.319412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1333d-865c-4a3d-8787-1b2e64283bf0 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:39.321094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-19d1333d-865c-4a3d-8787-1b2e64283bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:39.472938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1333d-865c-4a3d-8787-1b2e64283bf0 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:39.538174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:39.593669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fde62-c9c4-444b-8548-4f6f33341550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:39.594186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fde62-c9c4-444b-8548-4f6f33341550 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:39.594468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fde62-c9c4-444b-8548-4f6f33341550 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:39.594741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fde62-c9c4-444b-8548-4f6f33341550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:39.596409 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-000fde62-c9c4-444b-8548-4f6f33341550 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:39.690703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fde62-c9c4-444b-8548-4f6f33341550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:39.692861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fde62-c9c4-444b-8548-4f6f33341550 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:39.711318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 680/1376] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:41.682111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1093e-2575-40a4-8f4a-a6f0005e691c None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:41.682723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1093e-2575-40a4-8f4a-a6f0005e691c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:41.682723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1093e-2575-40a4-8f4a-a6f0005e691c None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:41.684028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 697/1377] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:41.689321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0670d40-026b-41ba-9015-ca12c10883eb None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:41.689785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0670d40-026b-41ba-9015-ca12c10883eb None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:41.690009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0670d40-026b-41ba-9015-ca12c10883eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:41.760687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b0670d40-026b-41ba-9015-ca12c10883eb None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:41.761932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b0670d40-026b-41ba-9015-ca12c10883eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:41.769917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:41.877211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 681/1378] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:42.170232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1093e-2575-40a4-8f4a-a6f0005e691c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:42.172029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-84a1093e-2575-40a4-8f4a-a6f0005e691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:42.339334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1093e-2575-40a4-8f4a-a6f0005e691c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:42.343043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:42.493437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9721d25-690f-4ba1-b77d-0bebdf06c2a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:42.493949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9721d25-690f-4ba1-b77d-0bebdf06c2a7 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:42.494224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9721d25-690f-4ba1-b77d-0bebdf06c2a7 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:42.494495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9721d25-690f-4ba1-b77d-0bebdf06c2a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:42.496138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b9721d25-690f-4ba1-b77d-0bebdf06c2a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:42.601285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9721d25-690f-4ba1-b77d-0bebdf06c2a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:42.602988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9721d25-690f-4ba1-b77d-0bebdf06c2a7 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:42.632448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 698/1379] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:44.636542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b2a74-52c0-44e1-82a9-cf46682c6e8b None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:44.637115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b2a74-52c0-44e1-82a9-cf46682c6e8b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:44.637115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b2a74-52c0-44e1-82a9-cf46682c6e8b None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:44.638576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 682/1380] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:44.644336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c891be-e53a-4562-9c64-bdcd7a9775f8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:44.644795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c891be-e53a-4562-9c64-bdcd7a9775f8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:44.644896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c891be-e53a-4562-9c64-bdcd7a9775f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:44.657197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:44.717707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-96c891be-e53a-4562-9c64-bdcd7a9775f8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:44.718920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-96c891be-e53a-4562-9c64-bdcd7a9775f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:44.850250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 699/1381] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:45.127969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b2a74-52c0-44e1-82a9-cf46682c6e8b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:45.129680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-db7b2a74-52c0-44e1-82a9-cf46682c6e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:45.287435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:45.300281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b2a74-52c0-44e1-82a9-cf46682c6e8b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:45.447456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884cd2-1cca-42e0-a018-4ea896f01640 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:45.447995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884cd2-1cca-42e0-a018-4ea896f01640 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:45.448283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884cd2-1cca-42e0-a018-4ea896f01640 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:45.448518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884cd2-1cca-42e0-a018-4ea896f01640 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:45.450200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5f884cd2-1cca-42e0-a018-4ea896f01640 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:45.548476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f884cd2-1cca-42e0-a018-4ea896f01640 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:45.550698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f884cd2-1cca-42e0-a018-4ea896f01640 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:45.562010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:45.574074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 683/1382] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:47.696778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ba2f-0782-4729-8ef8-8b8845295a6b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:47.697331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ba2f-0782-4729-8ef8-8b8845295a6b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:47.697456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ba2f-0782-4729-8ef8-8b8845295a6b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:47.698942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 700/1383] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:47.704970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b59640-92e7-4880-8d76-087005b83902 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:47.705403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b59640-92e7-4880-8d76-087005b83902 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:47.705710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b59640-92e7-4880-8d76-087005b83902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:47.789478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d8b59640-92e7-4880-8d76-087005b83902 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:47.790991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d8b59640-92e7-4880-8d76-087005b83902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:47.915189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 684/1384] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:48.203983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764ba2f-0782-4729-8ef8-8b8845295a6b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:48.205529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a764ba2f-0782-4729-8ef8-8b8845295a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:48.294710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:48.360444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764ba2f-0782-4729-8ef8-8b8845295a6b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:48.481354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90600095-78ba-4d3f-9846-6211bcc508a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:48.481928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90600095-78ba-4d3f-9846-6211bcc508a2 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:48.482258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90600095-78ba-4d3f-9846-6211bcc508a2 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:48.482548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90600095-78ba-4d3f-9846-6211bcc508a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:48.484358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-90600095-78ba-4d3f-9846-6211bcc508a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:48.543219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:48.585643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90600095-78ba-4d3f-9846-6211bcc508a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:48.587284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90600095-78ba-4d3f-9846-6211bcc508a2 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:48.606331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 701/1385] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:00:50.535977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb3b3-076b-45ab-8443-29a2a544ee8f None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:50.536588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb3b3-076b-45ab-8443-29a2a544ee8f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:50.536588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb3b3-076b-45ab-8443-29a2a544ee8f None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:50.537876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 685/1386] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:00:50.543643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c763b3b-3d9e-4a19-916b-01e763066d9e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:50.544131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c763b3b-3d9e-4a19-916b-01e763066d9e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:50.544286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c763b3b-3d9e-4a19-916b-01e763066d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:50.616607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-3c763b3b-3d9e-4a19-916b-01e763066d9e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:50.617785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3c763b3b-3d9e-4a19-916b-01e763066d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:50.722130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 702/1387] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:51.014039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb3b3-076b-45ab-8443-29a2a544ee8f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:51.017339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-dcfcb3b3-076b-45ab-8443-29a2a544ee8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:51.123557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:51.210394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb3b3-076b-45ab-8443-29a2a544ee8f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:51.341272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4689c-ad68-4a77-9e5a-f83b45abfcd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:51.341916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4689c-ad68-4a77-9e5a-f83b45abfcd4 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:51.342208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4689c-ad68-4a77-9e5a-f83b45abfcd4 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:51.342485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4689c-ad68-4a77-9e5a-f83b45abfcd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:51.344175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a7f4689c-ad68-4a77-9e5a-f83b45abfcd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:51.384476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:51.438246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4689c-ad68-4a77-9e5a-f83b45abfcd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:51.440458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4689c-ad68-4a77-9e5a-f83b45abfcd4 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:51.459521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 686/1388] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:53.519043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dfd19-5e30-4c55-a6b5-bbca161f96db None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:53.519043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dfd19-5e30-4c55-a6b5-bbca161f96db None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:53.519043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dfd19-5e30-4c55-a6b5-bbca161f96db None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:53.520433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 703/1389] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:53.526191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595bf95-5b29-4b05-816f-ec5a5a7a728a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:53.526621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595bf95-5b29-4b05-816f-ec5a5a7a728a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:53.526776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595bf95-5b29-4b05-816f-ec5a5a7a728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:53.595802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e595bf95-5b29-4b05-816f-ec5a5a7a728a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:53.597563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e595bf95-5b29-4b05-816f-ec5a5a7a728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:53.700951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 687/1390] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:53.994076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dfd19-5e30-4c55-a6b5-bbca161f96db None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:53.995737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ae9dfd19-5e30-4c55-a6b5-bbca161f96db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:54.029247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:54.145856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dfd19-5e30-4c55-a6b5-bbca161f96db None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:54.250080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:54.262315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abbf1c-c82c-40c5-80ef-98afc7559312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:54.262959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abbf1c-c82c-40c5-80ef-98afc7559312 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:54.263272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abbf1c-c82c-40c5-80ef-98afc7559312 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:54.263547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abbf1c-c82c-40c5-80ef-98afc7559312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:54.265144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-99abbf1c-c82c-40c5-80ef-98afc7559312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:54.355864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abbf1c-c82c-40c5-80ef-98afc7559312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:54.357461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abbf1c-c82c-40c5-80ef-98afc7559312 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:54.375769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 704/1391] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:54.870587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7ff76-1769-44a5-81b3-24335f057350 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:54.871166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7ff76-1769-44a5-81b3-24335f057350 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:54.871166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7ff76-1769-44a5-81b3-24335f057350 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:54.872728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 688/1392] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:54.877898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b009-145c-4c20-94aa-b67409277179 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:54.878358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b009-145c-4c20-94aa-b67409277179 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:54.878439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9b009-145c-4c20-94aa-b67409277179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:54.953564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a7a9b009-145c-4c20-94aa-b67409277179 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:54.954804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a7a9b009-145c-4c20-94aa-b67409277179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:55.021060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:55.066097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 705/1393] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:55.372623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e37288-49c6-4f8b-85fc-3d54e7af0073 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:55.372992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e37288-49c6-4f8b-85fc-3d54e7af0073 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:55.373345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e37288-49c6-4f8b-85fc-3d54e7af0073 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:55.374764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 689/1394] 10.222.0.94 () {54 vars in 967 bytes} [Mon Aug 29 19:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:55.380558 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88d4c-7dd2-4554-a248-08323fde01d0 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:55.380914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88d4c-7dd2-4554-a248-08323fde01d0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:55.381254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88d4c-7dd2-4554-a248-08323fde01d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:55.465199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-29e88d4c-7dd2-4554-a248-08323fde01d0 None None] MFA Rules not processed for user `1d9d979b6539460e82759320c3d96a79`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:55.466357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-29e88d4c-7dd2-4554-a248-08323fde01d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:55.547750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 706/1395] 10.222.0.94 () {58 vars in 1064 bytes} [Mon Aug 29 19:00:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:55.555451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e37288-49c6-4f8b-85fc-3d54e7af0073 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:55.556961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-93e37288-49c6-4f8b-85fc-3d54e7af0073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:55.571665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:55.665390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e37288-49c6-4f8b-85fc-3d54e7af0073 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:55.741019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934e2d8-015e-42ba-8acc-e513a8371937 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:55.741552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934e2d8-015e-42ba-8acc-e513a8371937 service glance] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:55.741830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934e2d8-015e-42ba-8acc-e513a8371937 service glance] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:55.742102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934e2d8-015e-42ba-8acc-e513a8371937 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:55.743750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3934e2d8-015e-42ba-8acc-e513a8371937 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:55.819469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:55.867650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934e2d8-015e-42ba-8acc-e513a8371937 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:55.869088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934e2d8-015e-42ba-8acc-e513a8371937 service glance] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:55.888171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 690/1396] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:56.356417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55718-b460-44b4-84fd-8a0a4e95eb86 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:56.356990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55718-b460-44b4-84fd-8a0a4e95eb86 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:56.356990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55718-b460-44b4-84fd-8a0a4e95eb86 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:56.358327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 707/1397] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:56.364214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:56.364501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:56.364818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:56.436990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:56.438213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:56.551392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 691/1398] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:56.640861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8890-9ca0-45d4-8dee-4502f1a62202 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:56.641152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8890-9ca0-45d4-8dee-4502f1a62202 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:56.641433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa8890-9ca0-45d4-8dee-4502f1a62202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:56.712069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e5fa8890-9ca0-45d4-8dee-4502f1a62202 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:56.713293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e5fa8890-9ca0-45d4-8dee-4502f1a62202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:56.722215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:56.807439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 708/1399] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:00:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:56.848902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:56.850156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:56.854431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ef68c-afa9-4d20-86a6-673c4c77e42f None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:56.854726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ef68c-afa9-4d20-86a6-673c4c77e42f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:56.855008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ef68c-afa9-4d20-86a6-673c4c77e42f None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:56.856398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 709/1400] 10.222.0.94 () {54 vars in 943 bytes} [Mon Aug 29 19:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:00:56.863313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695685f-6a28-4be7-b913-3901c97cb465 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:56.863477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695685f-6a28-4be7-b913-3901c97cb465 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:56.863733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695685f-6a28-4be7-b913-3901c97cb465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:56.882090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:56.981803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434d75-326a-4e38-8c32-3ca6a5f8e2ff None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:57.061814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7695685f-6a28-4be7-b913-3901c97cb465 None None] MFA Rules not processed for user `7a0e57abdb8d4966a2066a85bbfbc15e`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:57.063290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7695685f-6a28-4be7-b913-3901c97cb465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:57.102218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64408e-4b47-4c69-a90c-b74112b9b64d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:57.102828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64408e-4b47-4c69-a90c-b74112b9b64d service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:57.103135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64408e-4b47-4c69-a90c-b74112b9b64d service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:57.103412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64408e-4b47-4c69-a90c-b74112b9b64d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:57.105025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2e64408e-4b47-4c69-a90c-b74112b9b64d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:57.124272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:57.151087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:57.176109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 710/1401] 10.222.0.94 () {58 vars in 1040 bytes} [Mon Aug 29 19:00:56 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:00:57.185765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695685f-6a28-4be7-b913-3901c97cb465 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:57.197112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695685f-6a28-4be7-b913-3901c97cb465 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:00:57.203961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64408e-4b47-4c69-a90c-b74112b9b64d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:57.206404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64408e-4b47-4c69-a90c-b74112b9b64d service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:57.224957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 692/1402] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:00:57.307368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136edc4-2348-47f9-9b33-26e7098f8b00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:00:57.307992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136edc4-2348-47f9-9b33-26e7098f8b00 service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:57.308270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136edc4-2348-47f9-9b33-26e7098f8b00 service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:57.308541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136edc4-2348-47f9-9b33-26e7098f8b00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:57.310283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e136edc4-2348-47f9-9b33-26e7098f8b00 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:57.407977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136edc4-2348-47f9-9b33-26e7098f8b00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:00:57.410874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136edc4-2348-47f9-9b33-26e7098f8b00 service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:00:57.420389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 711/1403] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:00:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:00:59.237723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99fc7-62aa-40b6-8cea-48ae99415555 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:59.238467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99fc7-62aa-40b6-8cea-48ae99415555 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:59.238467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99fc7-62aa-40b6-8cea-48ae99415555 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:59.241264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 693/1404] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:00:59.259898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3375b-e9c2-46e7-a6f9-9b3a25aab362 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:00:59.259898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3375b-e9c2-46e7-a6f9-9b3a25aab362 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:00:59.259898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3375b-e9c2-46e7-a6f9-9b3a25aab362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:00:59.271984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:59.339554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c8a3375b-e9c2-46e7-a6f9-9b3a25aab362 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:00:59.340810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c8a3375b-e9c2-46e7-a6f9-9b3a25aab362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:59.483189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 712/1405] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:00:59.776964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99fc7-62aa-40b6-8cea-48ae99415555 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:00:59.778588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-92a99fc7-62aa-40b6-8cea-48ae99415555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:00:59.955403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:00:59.958377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99fc7-62aa-40b6-8cea-48ae99415555 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:00.100637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bee9b0-b715-4624-8528-8b372976b7e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:00.101382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bee9b0-b715-4624-8528-8b372976b7e8 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:00.101673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bee9b0-b715-4624-8528-8b372976b7e8 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:00.101949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bee9b0-b715-4624-8528-8b372976b7e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:00.103641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-67bee9b0-b715-4624-8528-8b372976b7e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:00.217972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bee9b0-b715-4624-8528-8b372976b7e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:00.219725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bee9b0-b715-4624-8528-8b372976b7e8 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:00.243111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 694/1406] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:01:02.271480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfa54d-1085-4a9a-9fa5-aaa19657ed95 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:02.272077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfa54d-1085-4a9a-9fa5-aaa19657ed95 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:02.272077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfa54d-1085-4a9a-9fa5-aaa19657ed95 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:02.275241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 713/1407] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:02.278997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e9bc4-d939-4d25-8cd8-cba0bcc1129b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:02.279508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e9bc4-d939-4d25-8cd8-cba0bcc1129b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:02.279861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e9bc4-d939-4d25-8cd8-cba0bcc1129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:02.284717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:02.361607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-de4e9bc4-d939-4d25-8cd8-cba0bcc1129b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:02.362938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-de4e9bc4-d939-4d25-8cd8-cba0bcc1129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:02.468122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 695/1408] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:02.762392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfa54d-1085-4a9a-9fa5-aaa19657ed95 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:02.763948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-fddfa54d-1085-4a9a-9fa5-aaa19657ed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:02.867218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:02.933250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfa54d-1085-4a9a-9fa5-aaa19657ed95 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:03.069357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5fc4b-19c2-46fd-a24e-4ed2381a9f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:03.070122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5fc4b-19c2-46fd-a24e-4ed2381a9f1e service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:03.070471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5fc4b-19c2-46fd-a24e-4ed2381a9f1e service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:03.070805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5fc4b-19c2-46fd-a24e-4ed2381a9f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:03.072553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dfe5fc4b-19c2-46fd-a24e-4ed2381a9f1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:03.129586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:03.175509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5fc4b-19c2-46fd-a24e-4ed2381a9f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:03.177108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5fc4b-19c2-46fd-a24e-4ed2381a9f1e service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:03.196590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 714/1409] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:12.465934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=64 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:13.193914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=23 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:19.725966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1e646-479e-40f3-a620-9e368745a6d2 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:19.726546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1e646-479e-40f3-a620-9e368745a6d2 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:19.726546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1e646-479e-40f3-a620-9e368745a6d2 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:19.727911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 696/1410] 10.222.0.94 () {54 vars in 918 bytes} [Mon Aug 29 19:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:19.733242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3acd8f-501f-48bf-91ff-75f47ce74ad6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:19.733809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3acd8f-501f-48bf-91ff-75f47ce74ad6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:19.733809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3acd8f-501f-48bf-91ff-75f47ce74ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:19.825617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9f3acd8f-501f-48bf-91ff-75f47ce74ad6 None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:19.827804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9f3acd8f-501f-48bf-91ff-75f47ce74ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:19.937943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 715/1411] 10.222.0.94 () {58 vars in 1015 bytes} [Mon Aug 29 19:01:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:19.952512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55911735-8d22-4c55-b843-328aab811934 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:19.952810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55911735-8d22-4c55-b843-328aab811934 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:19.953161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55911735-8d22-4c55-b843-328aab811934 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:19.954646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 697/1412] 10.222.0.94 () {54 vars in 918 bytes} [Mon Aug 29 19:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:19.959577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacb2da-b1f3-4d01-bca5-454828756044 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:19.959865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacb2da-b1f3-4d01-bca5-454828756044 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:19.960141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacb2da-b1f3-4d01-bca5-454828756044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:20.033793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:20.036373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0bacb2da-b1f3-4d01-bca5-454828756044 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:20.038157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0bacb2da-b1f3-4d01-bca5-454828756044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:20.155443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 716/1413] 10.222.0.94 () {58 vars in 1015 bytes} [Mon Aug 29 19:01:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:22.565628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1897765-431d-440d-a038-7ea0317b6d5d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:22.566201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1897765-431d-440d-a038-7ea0317b6d5d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:22.566201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1897765-431d-440d-a038-7ea0317b6d5d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:22.567575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:22.572880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89d7ba-c4be-437c-8dbf-8e02ec17ed02 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:22.573316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89d7ba-c4be-437c-8dbf-8e02ec17ed02 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:22.573423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89d7ba-c4be-437c-8dbf-8e02ec17ed02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:22.640215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-bf89d7ba-c4be-437c-8dbf-8e02ec17ed02 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:22.641944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bf89d7ba-c4be-437c-8dbf-8e02ec17ed02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:22.698099 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:22.741440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:23.040709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1897765-431d-440d-a038-7ea0317b6d5d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:23.042308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d1897765-431d-440d-a038-7ea0317b6d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:23.192866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1897765-431d-440d-a038-7ea0317b6d5d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:23.265332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:23.307497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80f053-16f3-482d-9855-3ea8d2056c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:23.308111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80f053-16f3-482d-9855-3ea8d2056c7f service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:23.308384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80f053-16f3-482d-9855-3ea8d2056c7f service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:23.308652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80f053-16f3-482d-9855-3ea8d2056c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:23.310270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ec80f053-16f3-482d-9855-3ea8d2056c7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:23.398643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80f053-16f3-482d-9855-3ea8d2056c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:23.400277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80f053-16f3-482d-9855-3ea8d2056c7f service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:23.420009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 699/1416] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:24.325973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c31ed1-166f-49af-a4d7-034117a26a40 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:24.326526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c31ed1-166f-49af-a4d7-034117a26a40 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:24.326526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c31ed1-166f-49af-a4d7-034117a26a40 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:24.328077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:01:24.333284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c4e78-2541-4880-894b-6f5258c9620c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:24.333715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c4e78-2541-4880-894b-6f5258c9620c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:24.333850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c4e78-2541-4880-894b-6f5258c9620c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:24.411860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-da0c4e78-2541-4880-894b-6f5258c9620c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:24.413079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-da0c4e78-2541-4880-894b-6f5258c9620c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:24.441068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:24.510038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:24.661585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c31ed1-166f-49af-a4d7-034117a26a40 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:24.663050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e4c31ed1-166f-49af-a4d7-034117a26a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:24.811201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c31ed1-166f-49af-a4d7-034117a26a40 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:24.852958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:24.929006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca4482-b1ea-4555-8b75-e32453298ea6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:24.929534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca4482-b1ea-4555-8b75-e32453298ea6 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:24.929814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca4482-b1ea-4555-8b75-e32453298ea6 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:24.930110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca4482-b1ea-4555-8b75-e32453298ea6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:24.931874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-adca4482-b1ea-4555-8b75-e32453298ea6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:25.022961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca4482-b1ea-4555-8b75-e32453298ea6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:25.024588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca4482-b1ea-4555-8b75-e32453298ea6 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:25.042748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 719/1419] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:25.927642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea804d-7eef-4117-b588-98016e996d63 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:25.928187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea804d-7eef-4117-b588-98016e996d63 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:25.928187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea804d-7eef-4117-b588-98016e996d63 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:25.929692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:25.934529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5eec1-d486-44e7-861c-ffe2a013989b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:25.934998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5eec1-d486-44e7-861c-ffe2a013989b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:25.935122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5eec1-d486-44e7-861c-ffe2a013989b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:25.990821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:26.013050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cdb5eec1-d486-44e7-861c-ffe2a013989b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:26.014271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cdb5eec1-d486-44e7-861c-ffe2a013989b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:26.111545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:26.262054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea804d-7eef-4117-b588-98016e996d63 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:26.263652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-91ea804d-7eef-4117-b588-98016e996d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:26.414555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea804d-7eef-4117-b588-98016e996d63 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:26.427653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:26.533951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b81d38-311d-408d-9514-9881315c1457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:26.534561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b81d38-311d-408d-9514-9881315c1457 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:26.534842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b81d38-311d-408d-9514-9881315c1457 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:26.535149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b81d38-311d-408d-9514-9881315c1457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:26.536771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c5b81d38-311d-408d-9514-9881315c1457 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:26.628805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b81d38-311d-408d-9514-9881315c1457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:26.630394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b81d38-311d-408d-9514-9881315c1457 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:26.648868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 702/1422] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:27.528670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a3542-d9b9-4a4f-8f27-608eb17fff85 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:27.529238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a3542-d9b9-4a4f-8f27-608eb17fff85 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:27.529238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a3542-d9b9-4a4f-8f27-608eb17fff85 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:27.531162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:27.536437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7065d4-1b99-4718-b822-8b3eebc09d75 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:27.536887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7065d4-1b99-4718-b822-8b3eebc09d75 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:27.537068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7065d4-1b99-4718-b822-8b3eebc09d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:27.556615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:27.618943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cb7065d4-1b99-4718-b822-8b3eebc09d75 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:27.620285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cb7065d4-1b99-4718-b822-8b3eebc09d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:27.720562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:27.872449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a3542-d9b9-4a4f-8f27-608eb17fff85 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:27.874481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1d8a3542-d9b9-4a4f-8f27-608eb17fff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:27.995969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:28.033882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a3542-d9b9-4a4f-8f27-608eb17fff85 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:28.148270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac7061-3600-48ea-bf49-6423905bdce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:28.148786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac7061-3600-48ea-bf49-6423905bdce8 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:28.149065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac7061-3600-48ea-bf49-6423905bdce8 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:28.149337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac7061-3600-48ea-bf49-6423905bdce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:28.150931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-47ac7061-3600-48ea-bf49-6423905bdce8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:28.226894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:28.241190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac7061-3600-48ea-bf49-6423905bdce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:28.242787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac7061-3600-48ea-bf49-6423905bdce8 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:28.260988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 722/1425] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:29.127767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a696a0-57ad-498f-9c2f-0e539c610585 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:29.128275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a696a0-57ad-498f-9c2f-0e539c610585 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:29.128275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a696a0-57ad-498f-9c2f-0e539c610585 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:29.129601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:29.134769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9a25-5fb6-449e-ac06-45841212bde8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:29.135175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9a25-5fb6-449e-ac06-45841212bde8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:29.135372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9a25-5fb6-449e-ac06-45841212bde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:29.213372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1cef9a25-5fb6-449e-ac06-45841212bde8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:29.214641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1cef9a25-5fb6-449e-ac06-45841212bde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:29.310651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:01:29.464768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a696a0-57ad-498f-9c2f-0e539c610585 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:29.466204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b4a696a0-57ad-498f-9c2f-0e539c610585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:29.554490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:29.618905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a696a0-57ad-498f-9c2f-0e539c610585 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:29.738257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438b8f1-e5aa-406b-be63-862d5f690ee9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:29.738782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438b8f1-e5aa-406b-be63-862d5f690ee9 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:29.739060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438b8f1-e5aa-406b-be63-862d5f690ee9 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:29.739358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438b8f1-e5aa-406b-be63-862d5f690ee9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:29.740955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9438b8f1-e5aa-406b-be63-862d5f690ee9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:29.791011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:29.833291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438b8f1-e5aa-406b-be63-862d5f690ee9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:29.834981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438b8f1-e5aa-406b-be63-862d5f690ee9 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:29.854484 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 705/1428] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:30.694501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffea9e-4137-44b5-bb41-6e162e4c036e None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:30.695035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffea9e-4137-44b5-bb41-6e162e4c036e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:30.695035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffea9e-4137-44b5-bb41-6e162e4c036e None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:30.696636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:30.702952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6358d5-b32a-4489-915b-269e951ffe4d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:30.703694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6358d5-b32a-4489-915b-269e951ffe4d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:30.703694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6358d5-b32a-4489-915b-269e951ffe4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:30.782341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-9c6358d5-b32a-4489-915b-269e951ffe4d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:30.783675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9c6358d5-b32a-4489-915b-269e951ffe4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:30.881078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:31.042186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffea9e-4137-44b5-bb41-6e162e4c036e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:31.043851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e6ffea9e-4137-44b5-bb41-6e162e4c036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:31.096117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:31.194806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffea9e-4137-44b5-bb41-6e162e4c036e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:31.308911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729b0a6-8f52-4e1a-94fa-f188ba7720f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:31.309524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729b0a6-8f52-4e1a-94fa-f188ba7720f2 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:31.309806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729b0a6-8f52-4e1a-94fa-f188ba7720f2 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:31.310081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729b0a6-8f52-4e1a-94fa-f188ba7720f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:31.312557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d729b0a6-8f52-4e1a-94fa-f188ba7720f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:31.320781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:31.405840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729b0a6-8f52-4e1a-94fa-f188ba7720f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:31.407598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729b0a6-8f52-4e1a-94fa-f188ba7720f2 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:31.426643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 725/1431] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:32.234722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1cc4a-b087-4bb4-9a43-0a174d31dfb0 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.235355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1cc4a-b087-4bb4-9a43-0a174d31dfb0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.235355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1cc4a-b087-4bb4-9a43-0a174d31dfb0 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.236638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:32.241676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda8931-b954-4d37-8210-afec7bbf9cdb None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.242077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda8931-b954-4d37-8210-afec7bbf9cdb None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.242258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda8931-b954-4d37-8210-afec7bbf9cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.324363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-adda8931-b954-4d37-8210-afec7bbf9cdb None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:32.325582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-adda8931-b954-4d37-8210-afec7bbf9cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.410021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:32.443782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:32.595977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1cc4a-b087-4bb4-9a43-0a174d31dfb0 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:32.597495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ecf1cc4a-b087-4bb4-9a43-0a174d31dfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.616320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:32.746716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1cc4a-b087-4bb4-9a43-0a174d31dfb0 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:32.839989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:32.866079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcb044-af5a-4010-9204-928ec681786f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:32.866614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcb044-af5a-4010-9204-928ec681786f service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:32.866891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcb044-af5a-4010-9204-928ec681786f service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:32.867189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcb044-af5a-4010-9204-928ec681786f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:32.868804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-21fcb044-af5a-4010-9204-928ec681786f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:32.959448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcb044-af5a-4010-9204-928ec681786f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:32.961045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcb044-af5a-4010-9204-928ec681786f service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:32.981262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 708/1434] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:33.786069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9148c2-f716-4dfd-9a16-e46fedc14f6b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.786666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9148c2-f716-4dfd-9a16-e46fedc14f6b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.786666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9148c2-f716-4dfd-9a16-e46fedc14f6b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.788033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:33.792856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f9a21-0225-4ec8-9630-c2e6b5cb66a0 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:33.793305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f9a21-0225-4ec8-9630-c2e6b5cb66a0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:33.793451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f9a21-0225-4ec8-9630-c2e6b5cb66a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:33.871567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-794f9a21-0225-4ec8-9630-c2e6b5cb66a0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:33.872833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-794f9a21-0225-4ec8-9630-c2e6b5cb66a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:33.914848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:33.970627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:01:34.120403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9148c2-f716-4dfd-9a16-e46fedc14f6b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:34.121923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bf9148c2-f716-4dfd-9a16-e46fedc14f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:34.266987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9148c2-f716-4dfd-9a16-e46fedc14f6b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:34.320948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:34.383987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01110ea-cb2b-4ba3-837c-8364f37e9671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:34.384508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01110ea-cb2b-4ba3-837c-8364f37e9671 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:34.384784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01110ea-cb2b-4ba3-837c-8364f37e9671 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:34.385052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01110ea-cb2b-4ba3-837c-8364f37e9671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:34.386632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e01110ea-cb2b-4ba3-837c-8364f37e9671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:34.475350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01110ea-cb2b-4ba3-837c-8364f37e9671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:34.476926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01110ea-cb2b-4ba3-837c-8364f37e9671 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:34.494513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 728/1437] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:35.300910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0709062-832f-4d91-a648-b37947aa6a15 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.301477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0709062-832f-4d91-a648-b37947aa6a15 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.301477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0709062-832f-4d91-a648-b37947aa6a15 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.302805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:35.307681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adc4fb-19a9-4b3b-8a63-c3cb241db33d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.308083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adc4fb-19a9-4b3b-8a63-c3cb241db33d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.308235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adc4fb-19a9-4b3b-8a63-c3cb241db33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.384713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-40adc4fb-19a9-4b3b-8a63-c3cb241db33d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:35.385922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-40adc4fb-19a9-4b3b-8a63-c3cb241db33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.387992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:35.484554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:35.634959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0709062-832f-4d91-a648-b37947aa6a15 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:35.636522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d0709062-832f-4d91-a648-b37947aa6a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.783830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0709062-832f-4d91-a648-b37947aa6a15 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:35.810820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:35.898814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af856ce-9810-41a9-ba83-f7b3c497f6df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:35.899452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af856ce-9810-41a9-ba83-f7b3c497f6df service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:35.899734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af856ce-9810-41a9-ba83-f7b3c497f6df service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:35.900005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af856ce-9810-41a9-ba83-f7b3c497f6df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:35.901613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5af856ce-9810-41a9-ba83-f7b3c497f6df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:35.990788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af856ce-9810-41a9-ba83-f7b3c497f6df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:35.992425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af856ce-9810-41a9-ba83-f7b3c497f6df service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:36.011283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 711/1440] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:37.845441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731d59-15b0-4caf-b142-146df28d5b30 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:37.845908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731d59-15b0-4caf-b142-146df28d5b30 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:37.846005 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731d59-15b0-4caf-b142-146df28d5b30 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:37.847425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:37.853983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824874b-f9d9-475a-8288-bf5f1e9bdc67 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:37.854461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824874b-f9d9-475a-8288-bf5f1e9bdc67 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:37.854949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824874b-f9d9-475a-8288-bf5f1e9bdc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:37.902025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:37.928427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c824874b-f9d9-475a-8288-bf5f1e9bdc67 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:37.929658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c824874b-f9d9-475a-8288-bf5f1e9bdc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:38.027122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:38.310877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f731d59-15b0-4caf-b142-146df28d5b30 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:38.312517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8f731d59-15b0-4caf-b142-146df28d5b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:38.416646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f731d59-15b0-4caf-b142-146df28d5b30 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:38.448895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:38.490944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3f11a-6dc0-4753-8a1b-53088edcd845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:38.491577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3f11a-6dc0-4753-8a1b-53088edcd845 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:38.491854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3f11a-6dc0-4753-8a1b-53088edcd845 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:38.492122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3f11a-6dc0-4753-8a1b-53088edcd845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:38.493704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-79d3f11a-6dc0-4753-8a1b-53088edcd845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:38.605980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3f11a-6dc0-4753-8a1b-53088edcd845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:38.607374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3f11a-6dc0-4753-8a1b-53088edcd845 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:38.625444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 731/1443] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:40.774266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097d744-65b5-4364-a7d5-301b5ec64203 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:40.774836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097d744-65b5-4364-a7d5-301b5ec64203 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:40.774836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097d744-65b5-4364-a7d5-301b5ec64203 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:40.776207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:40.782018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fadbf-aa78-47b7-b6fb-84937e58115f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:40.782454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fadbf-aa78-47b7-b6fb-84937e58115f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:40.782572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fadbf-aa78-47b7-b6fb-84937e58115f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:40.853424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ea3fadbf-aa78-47b7-b6fb-84937e58115f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:40.854673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ea3fadbf-aa78-47b7-b6fb-84937e58115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:40.869350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:40.959708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:41.242133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097d744-65b5-4364-a7d5-301b5ec64203 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:41.243698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7097d744-65b5-4364-a7d5-301b5ec64203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:41.348846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097d744-65b5-4364-a7d5-301b5ec64203 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:41.361100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:41.422398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02130f-c36a-4148-8bbf-8403bc5bbc89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:41.422999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02130f-c36a-4148-8bbf-8403bc5bbc89 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:41.423303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02130f-c36a-4148-8bbf-8403bc5bbc89 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:41.423574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02130f-c36a-4148-8bbf-8403bc5bbc89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:41.425170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7f02130f-c36a-4148-8bbf-8403bc5bbc89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:41.532770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02130f-c36a-4148-8bbf-8403bc5bbc89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:41.534117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02130f-c36a-4148-8bbf-8403bc5bbc89 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:41.552861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 714/1446] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:43.774512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d04058-f60b-49e9-87a9-a3d94010163b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:43.775107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d04058-f60b-49e9-87a9-a3d94010163b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:43.775107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d04058-f60b-49e9-87a9-a3d94010163b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:43.776491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:43.782739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da9e4-e3db-4236-8d9c-e4f9e04376dd None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:43.783183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da9e4-e3db-4236-8d9c-e4f9e04376dd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:43.783338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da9e4-e3db-4236-8d9c-e4f9e04376dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:43.834750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:43.856038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d39da9e4-e3db-4236-8d9c-e4f9e04376dd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:43.857299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d39da9e4-e3db-4236-8d9c-e4f9e04376dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:43.966276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:44.254656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d04058-f60b-49e9-87a9-a3d94010163b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:44.256248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-46d04058-f60b-49e9-87a9-a3d94010163b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:44.362096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d04058-f60b-49e9-87a9-a3d94010163b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:44.402999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:44.437131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03fdd7-4b48-4f5a-8b33-45e78e5b5c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:44.437745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03fdd7-4b48-4f5a-8b33-45e78e5b5c6b service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:44.438022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03fdd7-4b48-4f5a-8b33-45e78e5b5c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:44.438291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03fdd7-4b48-4f5a-8b33-45e78e5b5c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:44.439943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7f03fdd7-4b48-4f5a-8b33-45e78e5b5c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:44.548935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03fdd7-4b48-4f5a-8b33-45e78e5b5c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:44.550288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03fdd7-4b48-4f5a-8b33-45e78e5b5c6b service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:44.569200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 734/1449] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:46.604124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab0a7a-ee7d-4f70-bd2e-0f1808311aa4 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:46.604699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab0a7a-ee7d-4f70-bd2e-0f1808311aa4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:46.604699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab0a7a-ee7d-4f70-bd2e-0f1808311aa4 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:46.606027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:01:46.611427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca3e7e-c145-4ae0-9920-7d3fbfe77506 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:46.611834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca3e7e-c145-4ae0-9920-7d3fbfe77506 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:46.611980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca3e7e-c145-4ae0-9920-7d3fbfe77506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:46.681228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-50ca3e7e-c145-4ae0-9920-7d3fbfe77506 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:46.682968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-50ca3e7e-c145-4ae0-9920-7d3fbfe77506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:46.707129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:46.783270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:47.074396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab0a7a-ee7d-4f70-bd2e-0f1808311aa4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:47.076349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-52ab0a7a-ee7d-4f70-bd2e-0f1808311aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:47.181711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab0a7a-ee7d-4f70-bd2e-0f1808311aa4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:47.203512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:47.258694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bf7cb-addc-4c7f-81f8-97ce3863a86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:47.259331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bf7cb-addc-4c7f-81f8-97ce3863a86e service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:47.259613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bf7cb-addc-4c7f-81f8-97ce3863a86e service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:47.259886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bf7cb-addc-4c7f-81f8-97ce3863a86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:47.261489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-195bf7cb-addc-4c7f-81f8-97ce3863a86e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:47.371566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bf7cb-addc-4c7f-81f8-97ce3863a86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:47.372980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bf7cb-addc-4c7f-81f8-97ce3863a86e service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:47.390893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 717/1452] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:49.393241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543608-29f4-4e24-ad76-96d5bbe5e9f9 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:49.393805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543608-29f4-4e24-ad76-96d5bbe5e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:49.393805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543608-29f4-4e24-ad76-96d5bbe5e9f9 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:49.395165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:49.400531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c5613-8c5c-4faa-8720-8d58b26434a0 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:49.400919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c5613-8c5c-4faa-8720-8d58b26434a0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:49.401082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c5613-8c5c-4faa-8720-8d58b26434a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:49.465507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:49.469000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5e8c5613-8c5c-4faa-8720-8d58b26434a0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:49.470802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e8c5613-8c5c-4faa-8720-8d58b26434a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:49.569203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:49.854263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50543608-29f4-4e24-ad76-96d5bbe5e9f9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:49.855835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-50543608-29f4-4e24-ad76-96d5bbe5e9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:49.959938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50543608-29f4-4e24-ad76-96d5bbe5e9f9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:50.009367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:50.034621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e6ff7-925e-440c-8ba6-4cc3bd929241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:50.035265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e6ff7-925e-440c-8ba6-4cc3bd929241 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:50.035545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e6ff7-925e-440c-8ba6-4cc3bd929241 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:50.035843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e6ff7-925e-440c-8ba6-4cc3bd929241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:50.037450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ab8e6ff7-925e-440c-8ba6-4cc3bd929241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:50.146612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e6ff7-925e-440c-8ba6-4cc3bd929241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:50.147996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e6ff7-925e-440c-8ba6-4cc3bd929241 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:50.166954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 737/1455] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:52.134176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c4207-0aca-4c9d-a844-f93a7831f94e None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.134711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c4207-0aca-4c9d-a844-f93a7831f94e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.134711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c4207-0aca-4c9d-a844-f93a7831f94e None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.136098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:52.142083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0ea4-3f73-41ba-9b02-a962163c1366 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.142539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0ea4-3f73-41ba-9b02-a962163c1366 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.142637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0ea4-3f73-41ba-9b02-a962163c1366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.224995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-212b0ea4-3f73-41ba-9b02-a962163c1366 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:52.226188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-212b0ea4-3f73-41ba-9b02-a962163c1366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.254062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:52.322084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:01:52.617102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c4207-0aca-4c9d-a844-f93a7831f94e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:52.618663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-548c4207-0aca-4c9d-a844-f93a7831f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.723480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c4207-0aca-4c9d-a844-f93a7831f94e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:52.752281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:52.799525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1021d16-2d75-4f65-a1b5-188af570b688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:52.800143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1021d16-2d75-4f65-a1b5-188af570b688 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:52.800422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1021d16-2d75-4f65-a1b5-188af570b688 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:52.800694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1021d16-2d75-4f65-a1b5-188af570b688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:52.802304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a1021d16-2d75-4f65-a1b5-188af570b688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:52.915815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1021d16-2d75-4f65-a1b5-188af570b688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:52.917169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1021d16-2d75-4f65-a1b5-188af570b688 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:52.936518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 720/1458] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:54.966841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962b45-4de5-4b8f-96c2-7924a1cadc46 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:54.967616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962b45-4de5-4b8f-96c2-7924a1cadc46 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:54.967616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962b45-4de5-4b8f-96c2-7924a1cadc46 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:54.968834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:54.974081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365856fb-d637-4a0b-8f01-46b894e08e62 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:54.974498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365856fb-d637-4a0b-8f01-46b894e08e62 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:54.974641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365856fb-d637-4a0b-8f01-46b894e08e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.040505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-365856fb-d637-4a0b-8f01-46b894e08e62 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:55.041722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-365856fb-d637-4a0b-8f01-46b894e08e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.050834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:55.142635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:55.427596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962b45-4de5-4b8f-96c2-7924a1cadc46 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:55.429155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c9962b45-4de5-4b8f-96c2-7924a1cadc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.533842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962b45-4de5-4b8f-96c2-7924a1cadc46 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:55.592058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:55.610684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898e1f4-7b29-4ab3-baeb-e8ebc0f00c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:55.611325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e1f4-7b29-4ab3-baeb-e8ebc0f00c7f service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:55.611609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e1f4-7b29-4ab3-baeb-e8ebc0f00c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:55.611884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e1f4-7b29-4ab3-baeb-e8ebc0f00c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:55.613491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a898e1f4-7b29-4ab3-baeb-e8ebc0f00c7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:55.724876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898e1f4-7b29-4ab3-baeb-e8ebc0f00c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:55.726248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898e1f4-7b29-4ab3-baeb-e8ebc0f00c7f service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:55.744839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 740/1461] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:01:57.698315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b9bf9-4736-449b-8c24-f61759a94898 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.698881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b9bf9-4736-449b-8c24-f61759a94898 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.698881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b9bf9-4736-449b-8c24-f61759a94898 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.701141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:01:57.707159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d055d-8fee-46c6-8196-07dff3f62b1c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:57.707572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d055d-8fee-46c6-8196-07dff3f62b1c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:57.707775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d055d-8fee-46c6-8196-07dff3f62b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:57.775456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a58d055d-8fee-46c6-8196-07dff3f62b1c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:01:57.776673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a58d055d-8fee-46c6-8196-07dff3f62b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:57.815684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:57.877450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:01:58.164723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b9bf9-4736-449b-8c24-f61759a94898 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:01:58.166251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3a5b9bf9-4736-449b-8c24-f61759a94898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:58.268824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b9bf9-4736-449b-8c24-f61759a94898 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:01:58.307854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:01:58.345370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c632fcf-2142-4197-808c-0b3486faa39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:01:58.345973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c632fcf-2142-4197-808c-0b3486faa39f service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:01:58.346249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c632fcf-2142-4197-808c-0b3486faa39f service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:01:58.346515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c632fcf-2142-4197-808c-0b3486faa39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:01:58.348154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8c632fcf-2142-4197-808c-0b3486faa39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:01:58.456328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c632fcf-2142-4197-808c-0b3486faa39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:01:58.457684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c632fcf-2142-4197-808c-0b3486faa39f service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:01:58.475955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 723/1464] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:00.433669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376284e1-0823-49a8-abca-f1284cc876f6 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:00.434843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376284e1-0823-49a8-abca-f1284cc876f6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:00.434843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376284e1-0823-49a8-abca-f1284cc876f6 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:00.436517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:00.442245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9a2ad-63ba-4502-a893-4936fde2f1b0 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:00.442736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9a2ad-63ba-4502-a893-4936fde2f1b0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:00.442845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9a2ad-63ba-4502-a893-4936fde2f1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:00.516471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-79b9a2ad-63ba-4502-a893-4936fde2f1b0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:00.517607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-79b9a2ad-63ba-4502-a893-4936fde2f1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:00.537926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:00.622725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:00.906100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376284e1-0823-49a8-abca-f1284cc876f6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:00.907723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-376284e1-0823-49a8-abca-f1284cc876f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:01.025951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376284e1-0823-49a8-abca-f1284cc876f6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:01.100052 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:01.107700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c973d1-f4cc-4621-a085-acebbc8acc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:01.108313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c973d1-f4cc-4621-a085-acebbc8acc83 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:01.108781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c973d1-f4cc-4621-a085-acebbc8acc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:01.109053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c973d1-f4cc-4621-a085-acebbc8acc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:01.110672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-94c973d1-f4cc-4621-a085-acebbc8acc83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:01.229294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c973d1-f4cc-4621-a085-acebbc8acc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:01.230766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c973d1-f4cc-4621-a085-acebbc8acc83 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:01.249358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 743/1467] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:03.232753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb554-b5fa-4183-9931-6043d63d5845 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:03.233249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb554-b5fa-4183-9931-6043d63d5845 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:03.233361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb554-b5fa-4183-9931-6043d63d5845 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:03.234652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:03.239959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf521461-df08-4894-b8d4-1751b4d72f2f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:03.240393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf521461-df08-4894-b8d4-1751b4d72f2f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:03.240496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf521461-df08-4894-b8d4-1751b4d72f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:03.307862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cf521461-df08-4894-b8d4-1751b4d72f2f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:03.309731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cf521461-df08-4894-b8d4-1751b4d72f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:03.358536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:03.407617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:03.692691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eb554-b5fa-4183-9931-6043d63d5845 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:03.694310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-6e4eb554-b5fa-4183-9931-6043d63d5845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:03.799270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eb554-b5fa-4183-9931-6043d63d5845 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:03.848835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:03.875490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea55ff1-b4ee-4104-815b-f125e25a1174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:03.876099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea55ff1-b4ee-4104-815b-f125e25a1174 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:03.876379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea55ff1-b4ee-4104-815b-f125e25a1174 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:03.876652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea55ff1-b4ee-4104-815b-f125e25a1174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:03.878271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8ea55ff1-b4ee-4104-815b-f125e25a1174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:03.993711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea55ff1-b4ee-4104-815b-f125e25a1174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:03.995087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea55ff1-b4ee-4104-815b-f125e25a1174 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:04.014134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 726/1470] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:13.405760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=22 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:14.011191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=66 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:23.676807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447b55-5425-4dcc-9639-00657a96c5e8 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.677428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447b55-5425-4dcc-9639-00657a96c5e8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.677428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447b55-5425-4dcc-9639-00657a96c5e8 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.678750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 745/1471] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:23.684198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fedd7-80de-4c5b-9c38-dbdce784e9b6 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:23.684680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fedd7-80de-4c5b-9c38-dbdce784e9b6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:23.684796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fedd7-80de-4c5b-9c38-dbdce784e9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:23.755100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b79fedd7-80de-4c5b-9c38-dbdce784e9b6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:23.756353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b79fedd7-80de-4c5b-9c38-dbdce784e9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:23.853148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 727/1472] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:24.152800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc190fb-b5c6-4c77-a564-3516ebc02df3 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.153081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc190fb-b5c6-4c77-a564-3516ebc02df3 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.153354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc190fb-b5c6-4c77-a564-3516ebc02df3 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.154672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 746/1473] 10.222.0.94 () {54 vars in 967 bytes} [Mon Aug 29 19:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:24.159434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755402e-4316-4413-8a17-74dd17640706 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.159726 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755402e-4316-4413-8a17-74dd17640706 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.160006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755402e-4316-4413-8a17-74dd17640706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.237890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7755402e-4316-4413-8a17-74dd17640706 None None] MFA Rules not processed for user `1d9d979b6539460e82759320c3d96a79`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:24.239119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7755402e-4316-4413-8a17-74dd17640706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:24.260037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.316477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 728/1474] 10.222.0.94 () {58 vars in 1064 bytes} [Mon Aug 29 19:02:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:24.323956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc190fb-b5c6-4c77-a564-3516ebc02df3 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:24.325481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-edc190fb-b5c6-4c77-a564-3516ebc02df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:24.428653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc190fb-b5c6-4c77-a564-3516ebc02df3 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:24.501666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35e73-db77-4839-8f88-9e2ed4e315c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:24.502271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35e73-db77-4839-8f88-9e2ed4e315c8 service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:24.502544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35e73-db77-4839-8f88-9e2ed4e315c8 service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:24.502811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35e73-db77-4839-8f88-9e2ed4e315c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:24.504725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e4d35e73-db77-4839-8f88-9e2ed4e315c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:24.559320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:24.616145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35e73-db77-4839-8f88-9e2ed4e315c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:24.617621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35e73-db77-4839-8f88-9e2ed4e315c8 service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:24.637816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 747/1475] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:26.643984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6a8dd-79ac-45ba-aa94-38f1d71e7159 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:26.644517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6a8dd-79ac-45ba-aa94-38f1d71e7159 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:26.644517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6a8dd-79ac-45ba-aa94-38f1d71e7159 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:26.645783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 729/1476] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:26.651137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d04d4-eb8a-4f24-9f0d-620f9a1baf3e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:26.651580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d04d4-eb8a-4f24-9f0d-620f9a1baf3e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:26.651704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d04d4-eb8a-4f24-9f0d-620f9a1baf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:26.717743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d70d04d4-eb8a-4f24-9f0d-620f9a1baf3e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:26.718943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d70d04d4-eb8a-4f24-9f0d-620f9a1baf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:26.816486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 748/1477] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:27.089953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6a8dd-79ac-45ba-aa94-38f1d71e7159 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:27.091568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-76f6a8dd-79ac-45ba-aa94-38f1d71e7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:27.220587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6a8dd-79ac-45ba-aa94-38f1d71e7159 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:27.270592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:27.318019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bd667-ba66-4653-aa96-15aa13c1144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:27.318621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bd667-ba66-4653-aa96-15aa13c1144d admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:27.318894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bd667-ba66-4653-aa96-15aa13c1144d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:27.319189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bd667-ba66-4653-aa96-15aa13c1144d admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:27.319826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bd667-ba66-4653-aa96-15aa13c1144d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:27.321594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-6f5bd667-ba66-4653-aa96-15aa13c1144d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:02:27.321594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:02:27.321594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:02:27.322762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bd667-ba66-4653-aa96-15aa13c1144d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:27.342495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 730/1478] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Aug 29 19:02:27 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:02:29.274093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23824-c2b9-42c7-b2b3-699042058b40 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:29.274586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23824-c2b9-42c7-b2b3-699042058b40 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:29.274690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23824-c2b9-42c7-b2b3-699042058b40 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:29.276354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 749/1479] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:29.281921 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86525c2c-31b3-48cd-9a5d-9cf59f5c2edf None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:29.282365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86525c2c-31b3-48cd-9a5d-9cf59f5c2edf None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:29.282475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86525c2c-31b3-48cd-9a5d-9cf59f5c2edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:29.354983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-86525c2c-31b3-48cd-9a5d-9cf59f5c2edf None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:29.356250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-86525c2c-31b3-48cd-9a5d-9cf59f5c2edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:29.458221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 731/1480] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:29.750207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f23824-c2b9-42c7-b2b3-699042058b40 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:29.751812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b1f23824-c2b9-42c7-b2b3-699042058b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:29.763053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:29.902504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f23824-c2b9-42c7-b2b3-699042058b40 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:29.983656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:30.020816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc782e-eb63-43ce-81e8-862b22c39737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:30.021367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc782e-eb63-43ce-81e8-862b22c39737 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:30.021788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc782e-eb63-43ce-81e8-862b22c39737 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:30.022075 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc782e-eb63-43ce-81e8-862b22c39737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:30.023758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-abdc782e-eb63-43ce-81e8-862b22c39737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:30.113772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc782e-eb63-43ce-81e8-862b22c39737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:30.115393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc782e-eb63-43ce-81e8-862b22c39737 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:30.133818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 750/1481] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:32.247006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a24d2a-f7d5-4b4b-ad67-ec6560f14de6 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:32.247827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a24d2a-f7d5-4b4b-ad67-ec6560f14de6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:32.247827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a24d2a-f7d5-4b4b-ad67-ec6560f14de6 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:32.248949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 732/1482] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:32.254163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1e986-9b92-48d9-a01a-29ac2df17ffc None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:32.254574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1e986-9b92-48d9-a01a-29ac2df17ffc None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:32.254718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1e986-9b92-48d9-a01a-29ac2df17ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:32.324093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2bf1e986-9b92-48d9-a01a-29ac2df17ffc None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:32.325342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2bf1e986-9b92-48d9-a01a-29ac2df17ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:32.371131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:32.423969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 751/1483] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:32.717393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a24d2a-f7d5-4b4b-ad67-ec6560f14de6 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:32.718944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d2a24d2a-f7d5-4b4b-ad67-ec6560f14de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:32.742849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:32.823762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a24d2a-f7d5-4b4b-ad67-ec6560f14de6 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:32.898247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5479a6-85a2-4119-9af6-d27938a387fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:32.898861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5479a6-85a2-4119-9af6-d27938a387fd service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:32.899178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5479a6-85a2-4119-9af6-d27938a387fd service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:32.899459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5479a6-85a2-4119-9af6-d27938a387fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:32.901955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cb5479a6-85a2-4119-9af6-d27938a387fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:32.982888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:33.019637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5479a6-85a2-4119-9af6-d27938a387fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:33.021082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5479a6-85a2-4119-9af6-d27938a387fd service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:33.040199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 733/1484] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:35.022124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c493-4266-4d0b-9cb6-1b60555ac473 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.022690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c493-4266-4d0b-9cb6-1b60555ac473 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.022690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909c493-4266-4d0b-9cb6-1b60555ac473 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.024119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 752/1485] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:35.032259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612227c9-bd43-45ec-8df1-48a3ad02f6f7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.032723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612227c9-bd43-45ec-8df1-48a3ad02f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.032837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612227c9-bd43-45ec-8df1-48a3ad02f6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.101605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-612227c9-bd43-45ec-8df1-48a3ad02f6f7 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:35.102816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-612227c9-bd43-45ec-8df1-48a3ad02f6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.207106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 734/1486] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:02:35.490287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909c493-4266-4d0b-9cb6-1b60555ac473 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:35.491888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0909c493-4266-4d0b-9cb6-1b60555ac473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.597790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909c493-4266-4d0b-9cb6-1b60555ac473 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:35.677110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:35.679270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e472b-f6cb-4ed9-a5cb-0870abcd1ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:35.680008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e472b-f6cb-4ed9-a5cb-0870abcd1ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:35.680472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e472b-f6cb-4ed9-a5cb-0870abcd1ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:35.680788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e472b-f6cb-4ed9-a5cb-0870abcd1ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:35.684573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4b3e472b-f6cb-4ed9-a5cb-0870abcd1ff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:35.795025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e472b-f6cb-4ed9-a5cb-0870abcd1ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:35.796414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e472b-f6cb-4ed9-a5cb-0870abcd1ff1 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:35.815744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 753/1487] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:37.815462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e6af-ff1a-4d79-992a-adfd97ab81b9 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.816056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e6af-ff1a-4d79-992a-adfd97ab81b9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.816056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665e6af-ff1a-4d79-992a-adfd97ab81b9 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.817372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 735/1488] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:37.823309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b70e5-086c-4708-b44f-5484d3d0fc62 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:37.823758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b70e5-086c-4708-b44f-5484d3d0fc62 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:37.824046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b70e5-086c-4708-b44f-5484d3d0fc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:37.892580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c81b70e5-086c-4708-b44f-5484d3d0fc62 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:37.893804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c81b70e5-086c-4708-b44f-5484d3d0fc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:37.946368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:37.991109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 754/1489] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:38.286995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665e6af-ff1a-4d79-992a-adfd97ab81b9 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:38.288584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e665e6af-ff1a-4d79-992a-adfd97ab81b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.325479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.392410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665e6af-ff1a-4d79-992a-adfd97ab81b9 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:38.464022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82e172-9348-454d-a8dd-97022c5729f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:38.464631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e172-9348-454d-a8dd-97022c5729f4 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:38.464908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e172-9348-454d-a8dd-97022c5729f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:38.465179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e172-9348-454d-a8dd-97022c5729f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:38.466777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3c82e172-9348-454d-a8dd-97022c5729f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:38.549109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:38.578263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82e172-9348-454d-a8dd-97022c5729f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:38.579736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82e172-9348-454d-a8dd-97022c5729f4 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:38.600112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 736/1490] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:40.817940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4d2f0-93b3-48de-bfa5-9ffc281169c9 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.818530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4d2f0-93b3-48de-bfa5-9ffc281169c9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.818530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4d2f0-93b3-48de-bfa5-9ffc281169c9 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:40.819875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 755/1491] 10.222.0.94 () {54 vars in 927 bytes} [Mon Aug 29 19:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:02:40.825817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac0fd6-d63d-4305-a561-d2c2dae62581 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:40.826254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac0fd6-d63d-4305-a561-d2c2dae62581 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:40.826385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac0fd6-d63d-4305-a561-d2c2dae62581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.012513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-59ac0fd6-d63d-4305-a561-d2c2dae62581 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:02:41.013748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-59ac0fd6-d63d-4305-a561-d2c2dae62581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:41.113581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 737/1492] 10.222.0.94 () {58 vars in 1024 bytes} [Mon Aug 29 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:02:41.405169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4d2f0-93b3-48de-bfa5-9ffc281169c9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:02:41.406721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b2b4d2f0-93b3-48de-bfa5-9ffc281169c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:41.510704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4d2f0-93b3-48de-bfa5-9ffc281169c9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:02:41.588063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f3daa-18b2-4fba-adc3-03f89b3b5f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:02:41.588683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f3daa-18b2-4fba-adc3-03f89b3b5f9d service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:02:41.588961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f3daa-18b2-4fba-adc3-03f89b3b5f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:02:41.589235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f3daa-18b2-4fba-adc3-03f89b3b5f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:02:41.590844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-123f3daa-18b2-4fba-adc3-03f89b3b5f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:02:41.600067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:41.702313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f3daa-18b2-4fba-adc3-03f89b3b5f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:02:41.703713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f3daa-18b2-4fba-adc3-03f89b3b5f9d service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:02:41.722055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 756/1493] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:02:51.111814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=82 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:02:51.719821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=50 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:55.774562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.775205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.775205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:55.854557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:55.855818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:55.956587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 738/1494] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:55.964915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0b835-64b7-4e01-acae-5f846846e60b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:55.965491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0b835-64b7-4e01-acae-5f846846e60b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:55.965491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0b835-64b7-4e01-acae-5f846846e60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.048119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b7b0b835-64b7-4e01-acae-5f846846e60b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:56.049237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b7b0b835-64b7-4e01-acae-5f846846e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.147873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 757/1495] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:56.152779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.153998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.166030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34579b1a-34c7-4265-b7d4-3ce58f49096a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.228110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.262830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.263485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.263766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.264037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.264716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:03:56.265020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.266037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:56.266037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:03:56.266037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:03:56.266953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.280485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 739/1496] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:03:56.289646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0b835-64b7-4e01-acae-5f846846e60b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.299294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0b835-64b7-4e01-acae-5f846846e60b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.361323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.395645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.396247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.396523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.396793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.397537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.399547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING py.warnings [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:56.399547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: warnings.warn(msg) Aug 29 19:03:56.399547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: Aug 29 19:03:56.400749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.460984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 758/1497] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:03:56 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:03:56.465257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.466800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.502231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628700b-17e2-48b5-a2a6-128250ac71f2 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.598025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.598630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.598907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.599207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.599837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.601573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:56.601573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:03:56.601573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:03:56.602775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.645656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 740/1498] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:03:56 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:03:56.650632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.652184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.714284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2ddf4-ad28-4ecb-bea0-fb62af283f0f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.800806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.813495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.814102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.814383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.814653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.815326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.816360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING py.warnings [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:56.816360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: warnings.warn(msg) Aug 29 19:03:56.816360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: Aug 29 19:03:56.817252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.830215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 759/1499] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:56.834464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.836629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:56.848586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78294-c864-4e17-992d-aaaa38461520 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:56.875143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:56.947142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c7208-ad52-4ae0-97ae-976fb4cf2b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:56.947750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7208-ad52-4ae0-97ae-976fb4cf2b4f admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:56.948038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7208-ad52-4ae0-97ae-976fb4cf2b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:56.948306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c7208-ad52-4ae0-97ae-976fb4cf2b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:56.948948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c7208-ad52-4ae0-97ae-976fb4cf2b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:56.949949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-5a5c7208-ad52-4ae0-97ae-976fb4cf2b4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:56.949949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:03:56.949949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:03:56.950842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c7208-ad52-4ae0-97ae-976fb4cf2b4f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:56.963951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 741/1500] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:03:56.968183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:56.969727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.002961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1028623-7701-48ab-b5bc-eefab3476d2c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.097741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613d322-97f6-4e74-bc62-f3d4a962b026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.098397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613d322-97f6-4e74-bc62-f3d4a962b026 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.098727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613d322-97f6-4e74-bc62-f3d4a962b026 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.098949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613d322-97f6-4e74-bc62-f3d4a962b026 admin admin] PATH_INFO: `/v3/projects/8045d195081c4fd898c015c7eff1d3f7/users/b17eaddd537c4383bc93202fd6401b78/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.123779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613d322-97f6-4e74-bc62-f3d4a962b026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8045d195081c4fd898c015c7eff1d3f7, user_id=b17eaddd537c4383bc93202fd6401b78, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.124858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING py.warnings [None req-1613d322-97f6-4e74-bc62-f3d4a962b026 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:57.124858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: warnings.warn(msg) Aug 29 19:03:57.124858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: Aug 29 19:03:57.126910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613d322-97f6-4e74-bc62-f3d4a962b026 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.152701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 760/1501] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:03:56 2022] PUT /identity/v3/projects/8045d195081c4fd898c015c7eff1d3f7/users/b17eaddd537c4383bc93202fd6401b78/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:57.166173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adca910-91b0-400e-a51c-ee89eeca49d7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.166484 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adca910-91b0-400e-a51c-ee89eeca49d7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.166797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adca910-91b0-400e-a51c-ee89eeca49d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.224865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5adca910-91b0-400e-a51c-ee89eeca49d7 None None] MFA Rules not processed for user `b17eaddd537c4383bc93202fd6401b78`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:57.226620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5adca910-91b0-400e-a51c-ee89eeca49d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.307310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 742/1502] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:57.320504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1caf-1218-4521-985d-9c3dd7127987 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.320792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1caf-1218-4521-985d-9c3dd7127987 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.321071 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1caf-1218-4521-985d-9c3dd7127987 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.322410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 761/1503] 10.222.0.94 () {54 vars in 964 bytes} [Mon Aug 29 19:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:03:57.327182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e6cc1-a94e-44a5-8b54-c57544a070ec None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.327472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e6cc1-a94e-44a5-8b54-c57544a070ec None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.327770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e6cc1-a94e-44a5-8b54-c57544a070ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.359526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:57.405231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-095e6cc1-a94e-44a5-8b54-c57544a070ec None None] MFA Rules not processed for user `ee56cf453075431ab2791e15e803d51c`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:57.406428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-095e6cc1-a94e-44a5-8b54-c57544a070ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.482874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 743/1504] 10.222.0.94 () {58 vars in 1061 bytes} [Mon Aug 29 19:03:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:57.490600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b1caf-1218-4521-985d-9c3dd7127987 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.492696 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-988b1caf-1218-4521-985d-9c3dd7127987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.543616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:57.598790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b1caf-1218-4521-985d-9c3dd7127987 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:57.674444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:57.674962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.675271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.675544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.677154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.778944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:57.780419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:57.784103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:57.800026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 762/1505] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:03:57.847549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63216-e358-41e7-b2ec-285cc76e93a8 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:57.847974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63216-e358-41e7-b2ec-285cc76e93a8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:57.848099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d63216-e358-41e7-b2ec-285cc76e93a8 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:57.849363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 744/1506] 10.222.0.94 () {52 vars in 830 bytes} [Mon Aug 29 19:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:03:57.854070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:57.855342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:57.955323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad635-f9b2-4ae9-b547-172cd8e902d6 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:58.050819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:58.051550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:58.051741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:58.052010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] PATH_INFO: `/v3/users/b17eaddd537c4383bc93202fd6401b78` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:58.064272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17eaddd537c4383bc93202fd6401b78)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:58.065321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING py.warnings [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:58.065321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: warnings.warn(msg) Aug 29 19:03:58.065321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: Aug 29 19:03:58.066531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:58.076410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:58.136495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17eaddd537c4383bc93202fd6401b78'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.146870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17eaddd537c4383bc93202fd6401b78'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.147934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17eaddd537c4383bc93202fd6401b78'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.171756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17eaddd537c4383bc93202fd6401b78'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.186379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17eaddd537c4383bc93202fd6401b78'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:58.191908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 763/1507] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:03:57 2022] DELETE /identity/v3/users/b17eaddd537c4383bc93202fd6401b78 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:03:58.197067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:58.197356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:58.197637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:58.253986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:03:58.255142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:58.330495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:58.343439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 745/1508] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:03:58.352959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:58.354167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:58.454540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff27006-1dd4-453a-8ca7-abd8d3e060d1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:58.510457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:03:58.531749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372122c-9913-4095-8045-5ea3b2055277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:58.532366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372122c-9913-4095-8045-5ea3b2055277 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:58.532647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372122c-9913-4095-8045-5ea3b2055277 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:58.532918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372122c-9913-4095-8045-5ea3b2055277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:58.534522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2372122c-9913-4095-8045-5ea3b2055277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:58.643582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372122c-9913-4095-8045-5ea3b2055277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:58.645031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372122c-9913-4095-8045-5ea3b2055277 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:58.664830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 764/1509] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:03:59.129627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:03:59.131094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:03:59.143517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac059f9-dd84-4fa4-b9aa-1fbc411c4205 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:03:59.245237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:03:59.245867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:03:59.246145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:03:59.246416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] PATH_INFO: `/v3/projects/8045d195081c4fd898c015c7eff1d3f7` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:03:59.256264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8045d195081c4fd898c015c7eff1d3f7)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:03:59.257391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:03:59.257391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:03:59.257391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:03:59.258685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:03:59.319170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8045d195081c4fd898c015c7eff1d3f7', 'request_id': 'req-aead7158-b9ea-420f-993e-60b3f1153e83'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.333640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8045d195081c4fd898c015c7eff1d3f7', 'request_id': 'req-aead7158-b9ea-420f-993e-60b3f1153e83'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.343488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8045d195081c4fd898c015c7eff1d3f7', 'request_id': 'req-aead7158-b9ea-420f-993e-60b3f1153e83'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.367303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] The token cache is being invalidate because project 8045d195081c4fd898c015c7eff1d3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:03:59.367590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.368113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-aead7158-b9ea-420f-993e-60b3f1153e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:03:59.369455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 746/1510] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:03:59 2022] DELETE /identity/v3/projects/8045d195081c4fd898c015c7eff1d3f7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:04:08.661738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=62 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:04:09.364805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:05:37.536007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892725f8-4391-4d4b-8310-52d20d4ca03e None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:37.536625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892725f8-4391-4d4b-8310-52d20d4ca03e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:37.536625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892725f8-4391-4d4b-8310-52d20d4ca03e None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:37.537941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:05:37.543307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f7621-3c07-43bd-b108-ff14888dbf29 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:37.543718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f7621-3c07-43bd-b108-ff14888dbf29 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:37.543867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f7621-3c07-43bd-b108-ff14888dbf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:37.624772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-be8f7621-3c07-43bd-b108-ff14888dbf29 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:37.625913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-be8f7621-3c07-43bd-b108-ff14888dbf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:37.731861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:05:37.887042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892725f8-4391-4d4b-8310-52d20d4ca03e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:37.889221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-892725f8-4391-4d4b-8310-52d20d4ca03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:38.010959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892725f8-4391-4d4b-8310-52d20d4ca03e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:38.016570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b3f78-7cf5-4a07-920e-db9e46df3dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:38.017213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b3f78-7cf5-4a07-920e-db9e46df3dec service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:38.017555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b3f78-7cf5-4a07-920e-db9e46df3dec service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:38.017844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b3f78-7cf5-4a07-920e-db9e46df3dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:38.026104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b3f78-7cf5-4a07-920e-db9e46df3dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:38.027716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b3f78-7cf5-4a07-920e-db9e46df3dec service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:38.037425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 766/1513] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:38.872823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d688-f4a5-49be-bc67-0a526f846ca5 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:38.873336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d688-f4a5-49be-bc67-0a526f846ca5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:38.873445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d688-f4a5-49be-bc67-0a526f846ca5 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:38.874718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:05:38.880130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc373fe-3842-4860-881b-cb1b3c92063c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:38.880601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc373fe-3842-4860-881b-cb1b3c92063c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:38.880706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc373fe-3842-4860-881b-cb1b3c92063c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:38.901225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2bc373fe-3842-4860-881b-cb1b3c92063c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:38.902435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2bc373fe-3842-4860-881b-cb1b3c92063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:38.910780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:05:39.061913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed895-bf85-4d38-8e80-cd7c2ee2bc78 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:39.062202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed895-bf85-4d38-8e80-cd7c2ee2bc78 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:39.062510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed895-bf85-4d38-8e80-cd7c2ee2bc78 None None] PATH_INFO: `/v3` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:39.064063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 749/1516] 10.222.0.71 () {54 vars in 925 bytes} [Mon Aug 29 19:05:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:05:39.070400 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc373fe-3842-4860-881b-cb1b3c92063c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:39.077134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e08180-5c87-4aef-9f47-4edc342f3d7f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:39.077424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e08180-5c87-4aef-9f47-4edc342f3d7f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:39.077705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e08180-5c87-4aef-9f47-4edc342f3d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:39.086096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc373fe-3842-4860-881b-cb1b3c92063c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:39.092626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b60cf-b218-4e0a-8824-f034facf9ae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:39.093239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b60cf-b218-4e0a-8824-f034facf9ae6 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:39.093517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b60cf-b218-4e0a-8824-f034facf9ae6 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:39.093779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b60cf-b218-4e0a-8824-f034facf9ae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:39.104388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b60cf-b218-4e0a-8824-f034facf9ae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:39.105977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b60cf-b218-4e0a-8824-f034facf9ae6 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:39.116379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 768/1517] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:39.143190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-61e08180-5c87-4aef-9f47-4edc342f3d7f None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:39.144169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-61e08180-5c87-4aef-9f47-4edc342f3d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:39.179343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:05:39.226216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 750/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Aug 29 19:05:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:05:48.153311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc1f4d-244b-4409-bb48-68087474517e None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:48.153862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc1f4d-244b-4409-bb48-68087474517e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:48.153862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc1f4d-244b-4409-bb48-68087474517e None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:48.155127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 769/1519] 10.222.0.94 () {54 vars in 961 bytes} [Mon Aug 29 19:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:05:48.160992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ea85-ccb7-4509-8081-a66868234f19 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:48.161491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ea85-ccb7-4509-8081-a66868234f19 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:48.161595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ea85-ccb7-4509-8081-a66868234f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:48.180988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f815ea85-ccb7-4509-8081-a66868234f19 None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:48.182193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f815ea85-ccb7-4509-8081-a66868234f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:48.190835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 751/1520] 10.222.0.94 () {58 vars in 1058 bytes} [Mon Aug 29 19:05:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:05:48.198399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc1f4d-244b-4409-bb48-68087474517e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:48.213399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc1f4d-244b-4409-bb48-68087474517e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:48.218537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e559ab-b190-4918-9441-1a5271c80f30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:48.219194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e559ab-b190-4918-9441-1a5271c80f30 service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:48.219479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e559ab-b190-4918-9441-1a5271c80f30 service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:48.219757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e559ab-b190-4918-9441-1a5271c80f30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:48.228827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e559ab-b190-4918-9441-1a5271c80f30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:48.230388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e559ab-b190-4918-9441-1a5271c80f30 service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:48.240050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 770/1521] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:50.357072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef54cd5-b4b5-4f3e-8e18-dfe718cd411a None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:50.357648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef54cd5-b4b5-4f3e-8e18-dfe718cd411a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:50.357648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef54cd5-b4b5-4f3e-8e18-dfe718cd411a None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:50.358975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:05:50.363836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fddad6-6c78-4dde-ba94-a121b61c1890 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:50.364285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fddad6-6c78-4dde-ba94-a121b61c1890 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:50.364404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fddad6-6c78-4dde-ba94-a121b61c1890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:50.383524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-63fddad6-6c78-4dde-ba94-a121b61c1890 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:50.384710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-63fddad6-6c78-4dde-ba94-a121b61c1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:50.392824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:05:50.552418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef54cd5-b4b5-4f3e-8e18-dfe718cd411a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:50.567234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef54cd5-b4b5-4f3e-8e18-dfe718cd411a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:50.572803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924891-fa10-43c1-b4ac-77fa04a8a073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:50.573430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924891-fa10-43c1-b4ac-77fa04a8a073 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:50.573705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924891-fa10-43c1-b4ac-77fa04a8a073 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:50.573974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924891-fa10-43c1-b4ac-77fa04a8a073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:50.582984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924891-fa10-43c1-b4ac-77fa04a8a073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:50.584668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924891-fa10-43c1-b4ac-77fa04a8a073 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:50.594011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 753/1524] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:51.412845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8421105-68a3-4f58-93ff-6bd8c119e8de None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:51.413418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8421105-68a3-4f58-93ff-6bd8c119e8de None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:51.413418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8421105-68a3-4f58-93ff-6bd8c119e8de None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:51.414725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 19:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:05:51.419554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f82e85-0a25-4aff-9624-19544532279d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:51.419995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f82e85-0a25-4aff-9624-19544532279d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:51.420093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f82e85-0a25-4aff-9624-19544532279d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:51.440044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a4f82e85-0a25-4aff-9624-19544532279d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:51.441240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a4f82e85-0a25-4aff-9624-19544532279d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:51.449199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 19:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:05:51.610102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8421105-68a3-4f58-93ff-6bd8c119e8de None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:51.625401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8421105-68a3-4f58-93ff-6bd8c119e8de None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:51.631618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546a2cb-8b5a-4975-a2ba-7eaf34530e43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:51.632212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546a2cb-8b5a-4975-a2ba-7eaf34530e43 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:51.632492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546a2cb-8b5a-4975-a2ba-7eaf34530e43 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:51.632900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546a2cb-8b5a-4975-a2ba-7eaf34530e43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:51.640874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546a2cb-8b5a-4975-a2ba-7eaf34530e43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:51.643342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546a2cb-8b5a-4975-a2ba-7eaf34530e43 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:51.652671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 773/1527] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:55.807014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da0022-39d9-44ad-b761-bc9f1cc13c4a None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:55.807605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da0022-39d9-44ad-b761-bc9f1cc13c4a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:55.807605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da0022-39d9-44ad-b761-bc9f1cc13c4a None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:55.808945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:05:55.814204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14fef8-37ca-43fa-bd4b-b1b967dd3f86 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:55.814593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14fef8-37ca-43fa-bd4b-b1b967dd3f86 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:55.814751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14fef8-37ca-43fa-bd4b-b1b967dd3f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:55.835539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-3a14fef8-37ca-43fa-bd4b-b1b967dd3f86 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:55.837300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3a14fef8-37ca-43fa-bd4b-b1b967dd3f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:55.849154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:05:56.137912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da0022-39d9-44ad-b761-bc9f1cc13c4a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:56.139636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-55da0022-39d9-44ad-b761-bc9f1cc13c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:56.222509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da0022-39d9-44ad-b761-bc9f1cc13c4a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:56.226495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce905885-dfe6-4f09-a161-66fb84a0ea3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:56.227139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce905885-dfe6-4f09-a161-66fb84a0ea3c service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:56.227424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce905885-dfe6-4f09-a161-66fb84a0ea3c service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:56.227695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce905885-dfe6-4f09-a161-66fb84a0ea3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:56.236896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce905885-dfe6-4f09-a161-66fb84a0ea3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:56.238952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce905885-dfe6-4f09-a161-66fb84a0ea3c service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:56.248406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 756/1530] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:05:58.186076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7df3b-9ae7-4dd9-87f2-d937d330c3b9 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:58.186598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7df3b-9ae7-4dd9-87f2-d937d330c3b9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:58.186598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7df3b-9ae7-4dd9-87f2-d937d330c3b9 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:58.187901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:05:58.193874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24c72-edb2-4592-bf9c-f38ade5f1e9d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:58.194304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24c72-edb2-4592-bf9c-f38ade5f1e9d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:58.194398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24c72-edb2-4592-bf9c-f38ade5f1e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:58.214744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a6e24c72-edb2-4592-bf9c-f38ade5f1e9d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:05:58.215997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a6e24c72-edb2-4592-bf9c-f38ade5f1e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:05:58.223837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:05:58.510255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7df3b-9ae7-4dd9-87f2-d937d330c3b9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:05:58.521831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7df3b-9ae7-4dd9-87f2-d937d330c3b9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:05:58.525652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1b6f7-4d42-40c9-9c1a-54d7070f9cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:05:58.526248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b6f7-4d42-40c9-9c1a-54d7070f9cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:05:58.526523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b6f7-4d42-40c9-9c1a-54d7070f9cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:05:58.526789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b6f7-4d42-40c9-9c1a-54d7070f9cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:05:58.537041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1b6f7-4d42-40c9-9c1a-54d7070f9cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:05:58.538578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1b6f7-4d42-40c9-9c1a-54d7070f9cb2 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:05:58.547810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 776/1533] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:01.962615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce342616-af9a-4e4f-8fd8-48df71ec101b None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:01.963192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce342616-af9a-4e4f-8fd8-48df71ec101b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:01.963192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce342616-af9a-4e4f-8fd8-48df71ec101b None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:01.964527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:06:01.969790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219def84-1ca3-4055-b095-b30c519b6532 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:01.970191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219def84-1ca3-4055-b095-b30c519b6532 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:01.970362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219def84-1ca3-4055-b095-b30c519b6532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:01.978389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=87 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:01.991559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-219def84-1ca3-4055-b095-b30c519b6532 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:01.992769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-219def84-1ca3-4055-b095-b30c519b6532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:02.000636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:06:02.560757 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce342616-af9a-4e4f-8fd8-48df71ec101b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:02.562444 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ce342616-af9a-4e4f-8fd8-48df71ec101b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:02.644824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce342616-af9a-4e4f-8fd8-48df71ec101b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:02.648584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b7abe-549e-4130-a3d5-354bbafdf8a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:02.649300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7abe-549e-4130-a3d5-354bbafdf8a2 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:02.649579 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7abe-549e-4130-a3d5-354bbafdf8a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:02.649853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7abe-549e-4130-a3d5-354bbafdf8a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:02.657445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:02.661127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b7abe-549e-4130-a3d5-354bbafdf8a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:02.662882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b7abe-549e-4130-a3d5-354bbafdf8a2 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:02.673070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 759/1536] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:04.630996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4a48-fe00-4771-b439-8349cb2aa8ee None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:04.631600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4a48-fe00-4771-b439-8349cb2aa8ee None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:04.631600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4a48-fe00-4771-b439-8349cb2aa8ee None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:04.632926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 19:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:06:04.638263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ef44-ef7b-4bb1-8b8e-96a0734ee600 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:04.638657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ef44-ef7b-4bb1-8b8e-96a0734ee600 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:04.638812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ef44-ef7b-4bb1-8b8e-96a0734ee600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:04.659792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8e04ef44-ef7b-4bb1-8b8e-96a0734ee600 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:06:04.661196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8e04ef44-ef7b-4bb1-8b8e-96a0734ee600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:04.668958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 19:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:06:05.234997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4a48-fe00-4771-b439-8349cb2aa8ee None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:06:05.236700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d1fb4a48-fe00-4771-b439-8349cb2aa8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:06:05.249961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4a48-fe00-4771-b439-8349cb2aa8ee None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:06:05.253977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659411d4-c8bc-4e73-a8ce-a17a361209c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:06:05.254569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659411d4-c8bc-4e73-a8ce-a17a361209c9 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:06:05.254844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659411d4-c8bc-4e73-a8ce-a17a361209c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:06:05.255157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659411d4-c8bc-4e73-a8ce-a17a361209c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:06:05.263902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659411d4-c8bc-4e73-a8ce-a17a361209c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:06:05.265330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659411d4-c8bc-4e73-a8ce-a17a361209c9 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:06:05.275791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 779/1539] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:06:14.654513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:06:15.272475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=8 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:29.761021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad958-e8ed-4654-93e2-22ab415b33c6 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:29.762283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad958-e8ed-4654-93e2-22ab415b33c6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:29.762283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad958-e8ed-4654-93e2-22ab415b33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:29.800469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b81ad958-e8ed-4654-93e2-22ab415b33c6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:29.802367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b81ad958-e8ed-4654-93e2-22ab415b33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:29.813326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 761/1540] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:07:29.815365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557aafcb-0997-43f9-991c-ed6215b7f9f5 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:29.815893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557aafcb-0997-43f9-991c-ed6215b7f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:29.815893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557aafcb-0997-43f9-991c-ed6215b7f9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:29.827001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf0847-6142-49ae-a62f-86038f4b73a3 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:29.827410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf0847-6142-49ae-a62f-86038f4b73a3 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:29.827606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf0847-6142-49ae-a62f-86038f4b73a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:29.866654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-557aafcb-0997-43f9-991c-ed6215b7f9f5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:29.867927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-557aafcb-0997-43f9-991c-ed6215b7f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:29.881932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-68bf0847-6142-49ae-a62f-86038f4b73a3 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:29.883178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-68bf0847-6142-49ae-a62f-86038f4b73a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:29.898358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 780/1541] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:29.912862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 762/1542] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:29.918739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557aafcb-0997-43f9-991c-ed6215b7f9f5 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:29.922879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171d44a-8b89-4c8e-8921-06c0f72a2c39 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:29.923251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171d44a-8b89-4c8e-8921-06c0f72a2c39 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:29.923470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171d44a-8b89-4c8e-8921-06c0f72a2c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:29.936935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557aafcb-0997-43f9-991c-ed6215b7f9f5 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:29.942585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:29.943263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:29.943552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:29.943828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:29.944503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:29.944813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:29.945865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING py.warnings [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:07:29.945865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: warnings.warn(msg) Aug 29 19:07:29.945865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: Aug 29 19:07:29.951409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d13459-e54c-48a9-ba51-4ccb7c5674c4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:29.973164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-4171d44a-8b89-4c8e-8921-06c0f72a2c39 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:29.974617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 781/1543] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:07:29.975001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4171d44a-8b89-4c8e-8921-06c0f72a2c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:29.976093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9a905-f1dc-4544-b43d-0eef6663370d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:29.976559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9a905-f1dc-4544-b43d-0eef6663370d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:29.976849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9a905-f1dc-4544-b43d-0eef6663370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.002122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-91c9a905-f1dc-4544-b43d-0eef6663370d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.007549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-91c9a905-f1dc-4544-b43d-0eef6663370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.009709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 763/1544] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:07:30.010758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171d44a-8b89-4c8e-8921-06c0f72a2c39 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.023712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 782/1545] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.025253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9a905-f1dc-4544-b43d-0eef6663370d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.031926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171d44a-8b89-4c8e-8921-06c0f72a2c39 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.042255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9a905-f1dc-4544-b43d-0eef6663370d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.044376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.045009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.045288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.045561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.046788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.047026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5974d-201a-4c8d-9af1-d35b0df66fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.048006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5974d-201a-4c8d-9af1-d35b0df66fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.048299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5974d-201a-4c8d-9af1-d35b0df66fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.048828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5974d-201a-4c8d-9af1-d35b0df66fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.049496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5974d-201a-4c8d-9af1-d35b0df66fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:30.049909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5974d-201a-4c8d-9af1-d35b0df66fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.052060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5974d-201a-4c8d-9af1-d35b0df66fd9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.052314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:07:30.052314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:07:30.052314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:07:30.053506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.065094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 783/1546] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:07:30.066932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf703726-8139-4598-98b6-1d21fd383e60 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.067250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf703726-8139-4598-98b6-1d21fd383e60 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.067530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf703726-8139-4598-98b6-1d21fd383e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.091997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-bf703726-8139-4598-98b6-1d21fd383e60 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.093327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bf703726-8139-4598-98b6-1d21fd383e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.107170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 784/1547] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.108454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57045814-09fb-4a5d-b42a-0818c7527743 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.108747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57045814-09fb-4a5d-b42a-0818c7527743 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.109024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57045814-09fb-4a5d-b42a-0818c7527743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.141871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 764/1548] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/projects => generated 393 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.143179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.147577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-57045814-09fb-4a5d-b42a-0818c7527743 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.149278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-57045814-09fb-4a5d-b42a-0818c7527743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.230441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619dbf-8690-445a-9533-95662845c1c6 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.236525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4707478-8252-46ad-b2e0-1f8883fdbf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.237344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4707478-8252-46ad-b2e0-1f8883fdbf84 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.237685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4707478-8252-46ad-b2e0-1f8883fdbf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.237969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4707478-8252-46ad-b2e0-1f8883fdbf84 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.238686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4707478-8252-46ad-b2e0-1f8883fdbf84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.240794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4707478-8252-46ad-b2e0-1f8883fdbf84 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.241489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 785/1549] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.242336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57045814-09fb-4a5d-b42a-0818c7527743 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.251945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57045814-09fb-4a5d-b42a-0818c7527743 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.257555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.258165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.258475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.258750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.259658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:30.260045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.262133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.274254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 765/1550] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/projects => generated 403 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.275355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb123-5a2e-426e-8f6c-28b867dcb1fe None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.275809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 786/1551] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:30.276500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.279281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb123-5a2e-426e-8f6c-28b867dcb1fe None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.279601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb123-5a2e-426e-8f6c-28b867dcb1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.287986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba92c0-34cb-4c98-a4d3-d6ad9648d99c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.306124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7a4bb123-5a2e-426e-8f6c-28b867dcb1fe None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.307888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7a4bb123-5a2e-426e-8f6c-28b867dcb1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.352578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9e7a0-27aa-4b16-8a00-35039c3d249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.353186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9e7a0-27aa-4b16-8a00-35039c3d249a admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.353476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9e7a0-27aa-4b16-8a00-35039c3d249a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.353751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9e7a0-27aa-4b16-8a00-35039c3d249a admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.354390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9e7a0-27aa-4b16-8a00-35039c3d249a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.355433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING py.warnings [None req-bdf9e7a0-27aa-4b16-8a00-35039c3d249a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:07:30.355433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: warnings.warn(msg) Aug 29 19:07:30.355433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: Aug 29 19:07:30.356645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf9e7a0-27aa-4b16-8a00-35039c3d249a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.375151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 766/1552] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.375151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139f7d-51ce-48d0-a0bf-66f7ada3a520 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.375478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139f7d-51ce-48d0-a0bf-66f7ada3a520 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.375756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139f7d-51ce-48d0-a0bf-66f7ada3a520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.379847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 787/1553] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.381742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fc602-d8ca-4730-b774-5f711e9432c0 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.382037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fc602-d8ca-4730-b774-5f711e9432c0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.382317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fc602-d8ca-4730-b774-5f711e9432c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.399934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-2d139f7d-51ce-48d0-a0bf-66f7ada3a520 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.401167 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2d139f7d-51ce-48d0-a0bf-66f7ada3a520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.407448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2a1fc602-d8ca-4730-b774-5f711e9432c0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.408635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2a1fc602-d8ca-4730-b774-5f711e9432c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.409603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 767/1554] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.411719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead9d6a-9278-4ef6-9a60-379375a830ae None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.411851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead9d6a-9278-4ef6-9a60-379375a830ae None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.412121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead9d6a-9278-4ef6-9a60-379375a830ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.418247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 788/1555] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.419762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05836-9ed8-4fdb-8f3f-8e4f12d99da6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.420113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05836-9ed8-4fdb-8f3f-8e4f12d99da6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.421354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05836-9ed8-4fdb-8f3f-8e4f12d99da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.433435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8ead9d6a-9278-4ef6-9a60-379375a830ae None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.434656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8ead9d6a-9278-4ef6-9a60-379375a830ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.442453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-94b05836-9ed8-4fdb-8f3f-8e4f12d99da6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.442838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 768/1556] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.443671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead9d6a-9278-4ef6-9a60-379375a830ae None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.443803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-94b05836-9ed8-4fdb-8f3f-8e4f12d99da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.451999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 789/1557] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.452772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05836-9ed8-4fdb-8f3f-8e4f12d99da6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.455613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead9d6a-9278-4ef6-9a60-379375a830ae None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.462145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67f1d9-f2f2-475c-ac2e-dee675db4ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.462749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f1d9-f2f2-475c-ac2e-dee675db4ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.463048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f1d9-f2f2-475c-ac2e-dee675db4ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.463344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f1d9-f2f2-475c-ac2e-dee675db4ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.463662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05836-9ed8-4fdb-8f3f-8e4f12d99da6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.463973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67f1d9-f2f2-475c-ac2e-dee675db4ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.465902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67f1d9-f2f2-475c-ac2e-dee675db4ee5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.468375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.468967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.469247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.469520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.470145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.472100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.489470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:30.491185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 769/1558] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/users => generated 302 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.492427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750589-0b39-4efe-b086-365f86ea11a4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.492826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750589-0b39-4efe-b086-365f86ea11a4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.493165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44750589-0b39-4efe-b086-365f86ea11a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.503330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:30.503993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 790/1559] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.508489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.517010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-44750589-0b39-4efe-b086-365f86ea11a4 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.518213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-44750589-0b39-4efe-b086-365f86ea11a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.573858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccac67-f47c-4945-8aa5-020650f36202 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.578562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62c14c-94f9-47ed-8410-d378269a0d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.579202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c14c-94f9-47ed-8410-d378269a0d38 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.579623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c14c-94f9-47ed-8410-d378269a0d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.579732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 770/1560] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.579824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c14c-94f9-47ed-8410-d378269a0d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.580392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62c14c-94f9-47ed-8410-d378269a0d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.580875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43703-8139-45f3-b12a-cc42f118a7b0 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.581163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43703-8139-45f3-b12a-cc42f118a7b0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.581433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43703-8139-45f3-b12a-cc42f118a7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.582675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62c14c-94f9-47ed-8410-d378269a0d38 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.596119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 791/1561] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:30.596119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6fab5-708a-4f82-a4e5-bf48eeff6783 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.596119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6fab5-708a-4f82-a4e5-bf48eeff6783 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.596119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6fab5-708a-4f82-a4e5-bf48eeff6783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.603906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bed43703-8139-45f3-b12a-cc42f118a7b0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.605109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bed43703-8139-45f3-b12a-cc42f118a7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.612843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 771/1562] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.614398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69309bd1-1e11-4883-83a0-82036ec296be None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.614735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69309bd1-1e11-4883-83a0-82036ec296be None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.614937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f3d6fab5-708a-4f82-a4e5-bf48eeff6783 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.615245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69309bd1-1e11-4883-83a0-82036ec296be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.616692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f3d6fab5-708a-4f82-a4e5-bf48eeff6783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.624568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 792/1563] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.625373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6fab5-708a-4f82-a4e5-bf48eeff6783 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.635931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6fab5-708a-4f82-a4e5-bf48eeff6783 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.639050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-69309bd1-1e11-4883-83a0-82036ec296be None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:30.640271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-69309bd1-1e11-4883-83a0-82036ec296be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:30.640513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.641107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.641379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.641644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.642310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:30.642606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.644960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.649185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 772/1564] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:30.649931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69309bd1-1e11-4883-83a0-82036ec296be None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.658157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 793/1565] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:30.658937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.659355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69309bd1-1e11-4883-83a0-82036ec296be None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.665185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.665798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.666197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.666905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.667147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.668744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.670800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8319-493a-4972-bf79-431e79c88f36 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.676008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.676639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.676915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.677188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.677824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.679322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 773/1566] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:30.679782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.680128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.689778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b58a3-f3d9-4f50-a41a-1f713686bef7 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.695946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.696571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.696852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.697126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.697935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:30.698361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.700002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.707888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 794/1567] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.711954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.721636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 774/1568] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:30.721636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.721922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076e16d-de57-465b-9127-8099999156c7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.727210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.727816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.728101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.728371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.729132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.730723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.732509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64f5a-423e-4b03-8e35-32686fd0094e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.737205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.737800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.738089 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.738397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.739040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:30.739373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.741701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.742782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 795/1569] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:30.743627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.753235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bec56-ed9e-4196-8718-9bb22b1069d6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.758867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.759130 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 775/1570] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:30.759653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.759932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.760040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.760203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.760841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:30.761142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.762722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.771800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf852329-1619-41b9-9339-0e16e493be42 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.776030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 796/1571] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:30.776772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.777039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.777421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.777915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.778183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.778844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:30.779166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.780748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.788709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ead971-512f-45c1-94b7-05cfa56c66a1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.801902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 776/1572] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:30.802085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.802236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.802332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.802332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.802332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.802332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.802332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.808590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21755d1-ee3c-440e-a6e5-5c0a5c9b6f1a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.813358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.813960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.814236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.814512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.815220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.817866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.828524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 777/1573] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:30.829327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.829817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 797/1574] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.832878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.844296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bbd91-1951-4f95-93db-48616c860db1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.897946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d265241-cde0-46b0-8e76-7b532b5a9c79 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.901358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.901989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.902264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.902451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.902833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.903336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.903442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.903614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.903889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.904513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.905036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.906422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.915540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 778/1575] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:30.916341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.923641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:30.926888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c32ab-8526-4cd7-bb86-ffda98667903 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:30.932645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:30.933311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:30.933693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:30.933973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] PATH_INFO: `/v3/projects/b429eb37f4ed408d830ef2c1034e9bdc/users/73c4a7ab7fc7425389843798faa25372/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:30.938607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 798/1576] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:30.939443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.952366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b429eb37f4ed408d830ef2c1034e9bdc, user_id=73c4a7ab7fc7425389843798faa25372, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:30.953426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:07:30.953426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:07:30.953426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:07:30.955472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:30.975294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 779/1577] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:30 2022] PUT /identity/v3/projects/b429eb37f4ed408d830ef2c1034e9bdc/users/73c4a7ab7fc7425389843798faa25372/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:30.976043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:30.980358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:31.015517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a5804-b5ab-46d4-9542-70fc74f1b87b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.041483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7ebca-7bed-45d3-a906-a09528ba21bb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.046064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.046663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.046942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.047246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.047875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.049759 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.074563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.075215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.075496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.075770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.076401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.077194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 780/1578] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.078105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.078343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.106142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 799/1579] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:30 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.106717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.145705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02662174-8045-43f7-a59f-511a28eb9f64 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.170270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930830f4-7fe4-4407-acc2-75e87abf9cdf admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.174880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.176422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.176712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.176986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] PATH_INFO: `/v3/projects/39ae21f67cc649f1b573cdf7e4f97af8/users/0f83f56d35c5404a91185b1d140d484d/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.193938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ae21f67cc649f1b573cdf7e4f97af8, user_id=0f83f56d35c5404a91185b1d140d484d, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.196783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.206367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.207113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.207379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.207653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.208292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.216110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 800/1580] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:31 2022] PUT /identity/v3/projects/39ae21f67cc649f1b573cdf7e4f97af8/users/0f83f56d35c5404a91185b1d140d484d/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:31.216283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.216378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.225690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:31.243393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 781/1581] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.244662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.267795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:31.283293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121da6aa-94e8-42d3-afb3-4140ae098c6b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.310009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0e271-345a-4cc0-8b63-018b104b8910 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.314835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.315562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.315847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.316170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.317343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.318942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.328784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 782/1582] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.332838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.341427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280abc4-a2b3-4408-a48d-59d18759d317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.342038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280abc4-a2b3-4408-a48d-59d18759d317 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.342318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280abc4-a2b3-4408-a48d-59d18759d317 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.342585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280abc4-a2b3-4408-a48d-59d18759d317 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.343240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280abc4-a2b3-4408-a48d-59d18759d317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.343510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720aa7-8f8c-4d63-9409-94a43b293027 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.345145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280abc4-a2b3-4408-a48d-59d18759d317 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.348284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.348956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.349431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.349988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.350436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.352458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.368939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 801/1583] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.370164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc491f-5eb3-4ac1-a466-8de86cbcdbba None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.370454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc491f-5eb3-4ac1-a466-8de86cbcdbba None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.370715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc491f-5eb3-4ac1-a466-8de86cbcdbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.375736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 783/1584] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.376866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.388238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813a740-f834-4f93-a5f5-90b73745f665 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.390220 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-07fc491f-5eb3-4ac1-a466-8de86cbcdbba None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:31.391501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-07fc491f-5eb3-4ac1-a466-8de86cbcdbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:31.392986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b08fb-8407-4f71-9c75-ce35a2cacfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.393577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b08fb-8407-4f71-9c75-ce35a2cacfe9 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.393857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b08fb-8407-4f71-9c75-ce35a2cacfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.394134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b08fb-8407-4f71-9c75-ce35a2cacfe9 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.394954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b08fb-8407-4f71-9c75-ce35a2cacfe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.396901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b08fb-8407-4f71-9c75-ce35a2cacfe9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.399661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 802/1585] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:31.400420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc491f-5eb3-4ac1-a466-8de86cbcdbba None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.412277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc491f-5eb3-4ac1-a466-8de86cbcdbba None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.418652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.419235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.419235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.419559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.420156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.422068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.424807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 784/1586] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/users => generated 286 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.426580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd09036-cd63-486a-ba56-84d4a626f561 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.426855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd09036-cd63-486a-ba56-84d4a626f561 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.427152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd09036-cd63-486a-ba56-84d4a626f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.455708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 803/1587] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/users => generated 290 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.457796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.461248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bbd09036-cd63-486a-ba56-84d4a626f561 None None] MFA Rules not processed for user `0f83f56d35c5404a91185b1d140d484d`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:31.462833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bbd09036-cd63-486a-ba56-84d4a626f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:31.469484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fac7a-fa6b-4ce6-83f4-366a012f6b65 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.475862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.476564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.477021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.477356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.478057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.480087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.502215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 804/1588] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.504139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.514277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f68e1-e264-460d-af79-c9ea0d783b0f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.519001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.520293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.520574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.520852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] PATH_INFO: `/v3/projects/5dda60c835244f5f9c980ed1927d9784/users/624a0a90866446439918bd65ce5ddc60/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.535286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 785/1589] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:31.536257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd09036-cd63-486a-ba56-84d4a626f561 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.539857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dda60c835244f5f9c980ed1927d9784, user_id=624a0a90866446439918bd65ce5ddc60, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.542555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.546653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd09036-cd63-486a-ba56-84d4a626f561 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.552719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.553352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.553891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.554256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.554954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.560270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 805/1590] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:31 2022] PUT /identity/v3/projects/5dda60c835244f5f9c980ed1927d9784/users/624a0a90866446439918bd65ce5ddc60/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:31.560406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.560521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.568521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 786/1591] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.569447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.571304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:31.608090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:31.630258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9d0e4-7a6e-46e4-82e8-7824cf8cf462 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.634137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def012-44c2-43ed-97c0-c879557f3a18 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.634882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.635518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.635800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.636077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.636718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.638319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.647099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:31.649004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 806/1592] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.650015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.660525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983e93-6af3-457d-b5a5-a93803b5b351 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.665353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.666037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.666319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.666592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.667405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.669039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.679009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 807/1593] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.680208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.689570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540872cd-c0d7-4a59-8cca-552a9a2d7c83 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.694188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.694580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.694812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.695117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.695236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.695398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.695501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.695731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.696392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.697975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.704634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.706100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.709463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 808/1594] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.713157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.716979 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 787/1595] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:31.718420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.722889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d561b8-b1e4-4937-96a7-23ea85420521 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.727812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cd5e8-0a44-431f-a908-2ccf71048786 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.727961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefb1b9-e9f9-4168-86fb-4366d46b45f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.728444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefb1b9-e9f9-4168-86fb-4366d46b45f5 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.728680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefb1b9-e9f9-4168-86fb-4366d46b45f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.728951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefb1b9-e9f9-4168-86fb-4366d46b45f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.729591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefb1b9-e9f9-4168-86fb-4366d46b45f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.731218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefb1b9-e9f9-4168-86fb-4366d46b45f5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.733594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.733991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.734271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.734546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.735224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.737159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.750387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 809/1596] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.750387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053500-de12-4c0d-83b4-129b2b4909e0 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.750387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053500-de12-4c0d-83b4-129b2b4909e0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.750387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053500-de12-4c0d-83b4-129b2b4909e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.769160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cc053500-de12-4c0d-83b4-129b2b4909e0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:31.771521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cc053500-de12-4c0d-83b4-129b2b4909e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:31.773939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 788/1597] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.775393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.852051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 810/1598] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:31.852685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc053500-de12-4c0d-83b4-129b2b4909e0 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.864995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc053500-de12-4c0d-83b4-129b2b4909e0 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.867738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3ecd1-8c51-4a77-b261-537c3638acbd admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.870338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.871025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.871433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.871806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.872530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.873057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.873746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.874093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.874661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.874791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.875229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.876895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.890409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 811/1599] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.890409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.898907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 789/1600] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.900618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.902171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d28e5-4b1f-49e2-812b-ed2e8296e2b8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.912567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15624e58-53a5-4189-a7b3-9e6d89dd2920 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.912697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.913407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.913831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.914197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.915241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.916920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.917487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.919002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.919359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.919636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.920845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.923151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.931450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 812/1601] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.932766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.936660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 790/1602] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.940870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.946704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0b9ea-1efa-4b96-a8ec-b9d4427c7a92 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.951874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade85c80-102e-47d7-9c27-14d3bc8feae0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:31.952190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.952977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.953383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.953979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.954797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.956373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.958182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:31.958977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:31.959342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:31.959691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:31.960397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:31.962391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:31.971126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 813/1603] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:31.971398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:31.991018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 791/1604] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:31 2022] POST /identity/v3/users => generated 309 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:31.992416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.002909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f977e3-0e15-490a-9f1f-b71b2f775894 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.010116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.010886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.011260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.011643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] PATH_INFO: `/v3/projects/88c406ecbec9468487e33d84fd16afd2/users/e452d88233244dc8ada7372e7a30ef79/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.028445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c406ecbec9468487e33d84fd16afd2, user_id=e452d88233244dc8ada7372e7a30ef79, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.031271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.048279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fa876-7e0d-41ee-824b-a241bfa70f6b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.055087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 792/1605] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:31 2022] PUT /identity/v3/projects/88c406ecbec9468487e33d84fd16afd2/users/e452d88233244dc8ada7372e7a30ef79/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:32.055087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.092243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:32.127312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.127537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:32.127952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.128235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.128594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.154137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46335c-81f6-4ecf-a7f4-bb03aae8b83b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.160027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.161674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.161960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.162237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] PATH_INFO: `/v3/projects/b967c22662da47aeb9292704c39b44bb/users/ac0bc9e010d044a5a2147010cbdb4d85/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.179614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b967c22662da47aeb9292704c39b44bb, user_id=ac0bc9e010d044a5a2147010cbdb4d85, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.182319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.201732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 793/1606] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:32 2022] PUT /identity/v3/projects/b967c22662da47aeb9292704c39b44bb/users/ac0bc9e010d044a5a2147010cbdb4d85/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:32.202496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.216957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.219007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.230066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 814/1607] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:32.232174 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.244772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca9cb7-50f5-41a5-99fb-5620a36d3882 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.276069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb645e69-acad-42e5-9bb3-46bda0b7948e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.281300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.281984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.282283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.282565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] PATH_INFO: `/v3/projects/48587976e5aa4379ac6844050b072405/users/3cf9bdab54db49609ed37f819ce0ceb1/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.301259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48587976e5aa4379ac6844050b072405, user_id=3cf9bdab54db49609ed37f819ce0ceb1, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.304572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.317345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.318104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.318392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.318669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] PATH_INFO: `/v3/projects/74f3f1161ef84f30bbd787e9361a0dac/users/cc580d348f154c4488a5f84640539655/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.327171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 794/1608] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:32 2022] PUT /identity/v3/projects/48587976e5aa4379ac6844050b072405/users/3cf9bdab54db49609ed37f819ce0ceb1/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:32.327171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.345266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f3f1161ef84f30bbd787e9361a0dac, user_id=cc580d348f154c4488a5f84640539655, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.349798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.375320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 815/1609] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:32 2022] PUT /identity/v3/projects/74f3f1161ef84f30bbd787e9361a0dac/users/cc580d348f154c4488a5f84640539655/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:32.376048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.409589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670bffc-e6d7-45e7-8208-a6f6b126cfc9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.473171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e3651-9086-41ad-9248-d1556ecf35d7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.480103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:32.480434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca196ac-3be9-4053-bb4d-d13ed424fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.481079 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca196ac-3be9-4053-bb4d-d13ed424fe71 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.481348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca196ac-3be9-4053-bb4d-d13ed424fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.481692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca196ac-3be9-4053-bb4d-d13ed424fe71 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.482270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca196ac-3be9-4053-bb4d-d13ed424fe71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.484069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca196ac-3be9-4053-bb4d-d13ed424fe71 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.485694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:32.497224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153672e8-517d-498a-8c9b-dc18ce06435b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.497900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153672e8-517d-498a-8c9b-dc18ce06435b admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.498181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153672e8-517d-498a-8c9b-dc18ce06435b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.498455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153672e8-517d-498a-8c9b-dc18ce06435b admin admin] PATH_INFO: `/v3/projects/6f6085f7a1df49ae8e2375e28d7d6792/users/ff8ce50a95b742a48500e634db50fb36/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.502443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 816/1610] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:32.505433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5046e2-44ea-4858-82b3-b5480761d1b6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.505833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5046e2-44ea-4858-82b3-b5480761d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.506216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5046e2-44ea-4858-82b3-b5480761d1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.517793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153672e8-517d-498a-8c9b-dc18ce06435b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6085f7a1df49ae8e2375e28d7d6792, user_id=ff8ce50a95b742a48500e634db50fb36, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.521548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153672e8-517d-498a-8c9b-dc18ce06435b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.527981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-be5046e2-44ea-4858-82b3-b5480761d1b6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:32.529529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-be5046e2-44ea-4858-82b3-b5480761d1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:32.539276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 795/1611] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:32 2022] PUT /identity/v3/projects/6f6085f7a1df49ae8e2375e28d7d6792/users/ff8ce50a95b742a48500e634db50fb36/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:32.539276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4c105-fed7-40d3-9541-49f03f2daab8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.539276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4c105-fed7-40d3-9541-49f03f2daab8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.539729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4c105-fed7-40d3-9541-49f03f2daab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.561120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d9f4c105-fed7-40d3-9541-49f03f2daab8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:32.562351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d9f4c105-fed7-40d3-9541-49f03f2daab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:32.622293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 817/1612] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:32.624028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d765-8db8-4bc9-a522-e2a50e65a8ac None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.624269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d765-8db8-4bc9-a522-e2a50e65a8ac None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.624527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d765-8db8-4bc9-a522-e2a50e65a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.655865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 796/1613] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:32.657899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f3d88-4be8-4a57-984b-188ad518bf5b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.657899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f3d88-4be8-4a57-984b-188ad518bf5b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.658290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f3d88-4be8-4a57-984b-188ad518bf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.682850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cf55d765-8db8-4bc9-a522-e2a50e65a8ac None None] MFA Rules not processed for user `3cf9bdab54db49609ed37f819ce0ceb1`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:32.684673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cf55d765-8db8-4bc9-a522-e2a50e65a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:32.689771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1b8f3d88-4be8-4a57-984b-188ad518bf5b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:32.691852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1b8f3d88-4be8-4a57-984b-188ad518bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:32.724530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 797/1614] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:32.728843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f3d88-4be8-4a57-984b-188ad518bf5b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.748917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f3d88-4be8-4a57-984b-188ad518bf5b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.761238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.761902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.762215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.762552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] PATH_INFO: `/v3/projects/36fa703b5a7d4098ae0f4f7daadbeffb/users/c61a3ce640d24bb3aa33634b7da82dd8/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.793811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 818/1615] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:32.795790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb100ee-f519-486d-af8f-a5328230692f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.796242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb100ee-f519-486d-af8f-a5328230692f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.796692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb100ee-f519-486d-af8f-a5328230692f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:32.802503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36fa703b5a7d4098ae0f4f7daadbeffb, user_id=c61a3ce640d24bb3aa33634b7da82dd8, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:32.806124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:32.825366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-dcb100ee-f519-486d-af8f-a5328230692f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:32.826890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dcb100ee-f519-486d-af8f-a5328230692f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:32.838101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 798/1616] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:32 2022] PUT /identity/v3/projects/36fa703b5a7d4098ae0f4f7daadbeffb/users/c61a3ce640d24bb3aa33634b7da82dd8/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:32.839181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.900126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 819/1617] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:32.901129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb100ee-f519-486d-af8f-a5328230692f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:32.946402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:32.979323 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd925fc-d684-4335-ae41-80e29f7969c1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:32.989299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:32.990027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:32.991282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:32.991547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.004499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb100ee-f519-486d-af8f-a5328230692f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.012644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.014658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.015750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.016722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.017065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.020661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.043832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.045105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 799/1618] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:33.045978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.047082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.065633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eba91-9271-46bf-8828-09e7ec5b8582 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.068540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 820/1619] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:33.071271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.074798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.075912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.076195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.076472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.093536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce423-7531-43e5-a65a-a274d5e658f9 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.099712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.101377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.107294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.107926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.108265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.108548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.110093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.117322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.118713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 800/1620] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:33.120552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.135601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19940a1-eefe-473e-8e5c-f3ab59897e44 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.145201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.146467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.147210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.147515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.149104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.151014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.180059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 801/1621] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:33.180285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 821/1622] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:33 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:33.180398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.180492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.195268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec374-9fdc-4abf-a47c-1fa33b54b4d1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.235343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:33.325342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607402a-909e-4a69-95ad-378b0bbddca3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.340138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c567c87-2a38-4268-bb3f-147c3098cbc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.340865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c567c87-2a38-4268-bb3f-147c3098cbc3 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.341152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c567c87-2a38-4268-bb3f-147c3098cbc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.341500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c567c87-2a38-4268-bb3f-147c3098cbc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.343168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.344101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.344386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.345604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] PATH_INFO: `/v3/projects/36fa703b5a7d4098ae0f4f7daadbeffb/users/c61a3ce640d24bb3aa33634b7da82dd8/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.349218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36fa703b5a7d4098ae0f4f7daadbeffb, user_id=c61a3ce640d24bb3aa33634b7da82dd8, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.353988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.378303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c567c87-2a38-4268-bb3f-147c3098cbc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.382793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c567c87-2a38-4268-bb3f-147c3098cbc3 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.388716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 822/1623] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:33 2022] PUT /identity/v3/projects/36fa703b5a7d4098ae0f4f7daadbeffb/users/c61a3ce640d24bb3aa33634b7da82dd8/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:07:33.390119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.421321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 802/1624] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:33.424103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af8f0d-dcbe-445b-9208-bde5e365b812 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.425742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af8f0d-dcbe-445b-9208-bde5e365b812 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.426528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af8f0d-dcbe-445b-9208-bde5e365b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.501544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-80af8f0d-dcbe-445b-9208-bde5e365b812 None None] MFA Rules not processed for user `c61a3ce640d24bb3aa33634b7da82dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:33.502911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-80af8f0d-dcbe-445b-9208-bde5e365b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:33.556178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035302a0-1658-4e59-b8c7-bba9c5798f4d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.564851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.565764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.566341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.567503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.568342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.571348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.620535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 823/1625] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:33 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:33.622006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.650773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 803/1626] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:33.661756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37a993-39ee-4309-b3ef-40b194329b5d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.663605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af8f0d-dcbe-445b-9208-bde5e365b812 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.678737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.679614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.679901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.680176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.680837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.687897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.712080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 824/1627] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:33.716270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.743588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692713-4aba-41ab-8703-679ca43d749a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.755703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af8f0d-dcbe-445b-9208-bde5e365b812 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.760924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.761629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.761752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.762005 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.762195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] PATH_INFO: `/v3/projects/7fed10bc27eb4c4bbbb7526eeb36c89e/users/c7371bc7f7c8480e91f03987148a4b03/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.763554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.764156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.765777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.780564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.784393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.800388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 804/1628] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:33 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:33.804644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fed10bc27eb4c4bbbb7526eeb36c89e, user_id=c7371bc7f7c8480e91f03987148a4b03, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.807717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:33.814683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:33.862650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:33.874740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 825/1629] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:33 2022] PUT /identity/v3/projects/7fed10bc27eb4c4bbbb7526eeb36c89e/users/c7371bc7f7c8480e91f03987148a4b03/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:33.882672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08285418-e0bd-4717-9df8-412bc8513ecb service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:33.976157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:33.987760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a6a33-ac25-4010-b097-00d3ff016807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:33.989248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a6a33-ac25-4010-b097-00d3ff016807 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:33.989524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a6a33-ac25-4010-b097-00d3ff016807 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:33.990027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a6a33-ac25-4010-b097-00d3ff016807 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:33.991471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a6a33-ac25-4010-b097-00d3ff016807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:33.994397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a6a33-ac25-4010-b097-00d3ff016807 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:34.009757 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 805/1630] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:34.015055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:34.038053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69f303-245c-43ef-90c5-898f20a8c3a8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:34.045066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:34.046841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:34.047018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:34.047559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] PATH_INFO: `/v3/projects/7fed10bc27eb4c4bbbb7526eeb36c89e/users/c7371bc7f7c8480e91f03987148a4b03/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:34.059605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fed10bc27eb4c4bbbb7526eeb36c89e, user_id=c7371bc7f7c8480e91f03987148a4b03, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:34.065286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:34.120022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 826/1631] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:34 2022] PUT /identity/v3/projects/7fed10bc27eb4c4bbbb7526eeb36c89e/users/c7371bc7f7c8480e91f03987148a4b03/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:34.131910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c9e4d-c487-4aa5-8609-75f3d1e877c1 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:34.132284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c9e4d-c487-4aa5-8609-75f3d1e877c1 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:34.132567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c9e4d-c487-4aa5-8609-75f3d1e877c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:34.159619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:34.236272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-433c9e4d-c487-4aa5-8609-75f3d1e877c1 None None] MFA Rules not processed for user `c7371bc7f7c8480e91f03987148a4b03`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:34.237575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-433c9e4d-c487-4aa5-8609-75f3d1e877c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:34.301407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b74d8a-230c-4867-9541-ae34a9490a22 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:34.318850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:34.319738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:34.320114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:34.320501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] PATH_INFO: `/v3/users/0f83f56d35c5404a91185b1d140d484d` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:34.321814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f83f56d35c5404a91185b1d140d484d)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:34.323886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:34.359045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 806/1632] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:34.369723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c9e4d-c487-4aa5-8609-75f3d1e877c1 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:34.504926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f83f56d35c5404a91185b1d140d484d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:34.504926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f83f56d35c5404a91185b1d140d484d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:34.504926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f83f56d35c5404a91185b1d140d484d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:34.504926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f83f56d35c5404a91185b1d140d484d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:34.504926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-31556743-4ed2-4a24-9b66-fa49bde0044d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f83f56d35c5404a91185b1d140d484d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:34.517737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 827/1633] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:07:34 2022] DELETE /identity/v3/users/0f83f56d35c5404a91185b1d140d484d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:07:34.524004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:34.524407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:34.524940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:34.542235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c9e4d-c487-4aa5-8609-75f3d1e877c1 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:34.577422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:34.578689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:34.633155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:34.633834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:34.634121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:34.634392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:34.636727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:34.693347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 828/1634] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:07:34.735880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:34.761540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:34.764171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:34.787351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 807/1635] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:34.860850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:34.868891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:34.907256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:35.038611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:35.040631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:35.041898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e2460-ff62-4c7c-9cde-870d4a7fc928 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:35.045687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a933cf9-4259-4511-ae5b-c00f3264f02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:35.046301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a933cf9-4259-4511-ae5b-c00f3264f02e service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:35.046578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a933cf9-4259-4511-ae5b-c00f3264f02e service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:35.046856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a933cf9-4259-4511-ae5b-c00f3264f02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:35.053382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf875e42-435a-419b-98f2-9c132922d2c2 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:35.057687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:35.058774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:35.059275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:35.059718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:35.060054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a933cf9-4259-4511-ae5b-c00f3264f02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:35.061951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a933cf9-4259-4511-ae5b-c00f3264f02e service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:35.062461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:35.076162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 829/1636] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:35.201964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:35.203637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:35.218212 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 808/1637] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:36.325117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae787ae-acbe-4229-b150-9111685061d8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:36.325695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae787ae-acbe-4229-b150-9111685061d8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:36.325695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae787ae-acbe-4229-b150-9111685061d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:36.402378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-bae787ae-acbe-4229-b150-9111685061d8 None None] MFA Rules not processed for user `e452d88233244dc8ada7372e7a30ef79`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:36.404156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bae787ae-acbe-4229-b150-9111685061d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:36.589471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 830/1638] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:36.593621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:36.595214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:36.618275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d614cce-2e65-4fc9-860a-1f600a37ca93 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:36.629775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:36.630857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:36.631374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:36.631932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:36.632954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:36.639388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:36.721976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 809/1639] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:36 2022] POST /identity/v3/projects => generated 401 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:36.724068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae787ae-acbe-4229-b150-9111685061d8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:36.747558 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae787ae-acbe-4229-b150-9111685061d8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:36.957970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:36.958713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:36.958953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:36.959265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:36.959913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:36.962234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:37.002139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 831/1640] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:36 2022] POST /identity/v3/users => generated 307 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:37.004324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:37.063498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c11ce-6c55-4c7d-bc03-9fcf95210779 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:37.063498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:37.063498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:37.063498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:37.064288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:37.064288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:37.064288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:37.088002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:37.088937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 810/1641] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:37.098895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fabde-d8ea-478d-aea9-f0fcf99f89fa admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:37.126272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:37.126905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:37.127210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:37.127489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] PATH_INFO: `/v3/projects/f00fed15633e43d8a7df880cf9a62213/users/215c643b0f074070a3ad68ec2459e44f/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:37.166231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00fed15633e43d8a7df880cf9a62213, user_id=215c643b0f074070a3ad68ec2459e44f, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:37.175082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:37.207594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:37.212982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 832/1642] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:37 2022] PUT /identity/v3/projects/f00fed15633e43d8a7df880cf9a62213/users/215c643b0f074070a3ad68ec2459e44f/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:37.218002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:37.242103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780fe82-482f-41fa-a39e-aaad750a591c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:37.433997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:37.434305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:37.434591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:37.434864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:37.435549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:37.437195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:37.459899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 811/1643] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:37.466313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:37.486226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3fac2-84f1-47d0-a7c9-01235d94f561 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:37.507914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:37.510055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:37.510414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:37.514724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] PATH_INFO: `/v3/projects/f00fed15633e43d8a7df880cf9a62213/users/215c643b0f074070a3ad68ec2459e44f/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:37.521982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00fed15633e43d8a7df880cf9a62213, user_id=215c643b0f074070a3ad68ec2459e44f, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:37.531770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:37.555406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 833/1644] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:37 2022] PUT /identity/v3/projects/f00fed15633e43d8a7df880cf9a62213/users/215c643b0f074070a3ad68ec2459e44f/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:38.170494 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:38.172439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:38.198886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b1b46-6ee6-4a98-877d-93037516cebd admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:38.288807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:38.327536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:38.328116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:38.328399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:38.328678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] PATH_INFO: `/v3/projects/39ae21f67cc649f1b573cdf7e4f97af8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:38.333732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ae21f67cc649f1b573cdf7e4f97af8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:38.335721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:38.466405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ae21f67cc649f1b573cdf7e4f97af8', 'request_id': 'req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.487735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ae21f67cc649f1b573cdf7e4f97af8', 'request_id': 'req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.514322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ae21f67cc649f1b573cdf7e4f97af8', 'request_id': 'req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.535338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] The token cache is being invalidate because project 39ae21f67cc649f1b573cdf7e4f97af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:07:38.535759 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.536473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-44e9f18e-ebcf-4aae-b9f9-85bed0b74ade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.545151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 812/1645] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:07:38 2022] DELETE /identity/v3/projects/39ae21f67cc649f1b573cdf7e4f97af8 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:07:38.547613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:38.550830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:38.571833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e778e-9ae9-4fd1-9bbc-3639e210cc5c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:38.657470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:38.658390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:38.658810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:38.659403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] PATH_INFO: `/v3/users/c61a3ce640d24bb3aa33634b7da82dd8` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:38.661748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61a3ce640d24bb3aa33634b7da82dd8)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:38.663909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:38.772733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61a3ce640d24bb3aa33634b7da82dd8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.798258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61a3ce640d24bb3aa33634b7da82dd8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.799312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61a3ce640d24bb3aa33634b7da82dd8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.836656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61a3ce640d24bb3aa33634b7da82dd8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.846116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61a3ce640d24bb3aa33634b7da82dd8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:38.857223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 834/1646] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:07:38 2022] DELETE /identity/v3/users/c61a3ce640d24bb3aa33634b7da82dd8 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:39.809453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcdc05-ddc1-4c9a-8eaf-1e942a11df85 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:39.809947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcdc05-ddc1-4c9a-8eaf-1e942a11df85 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:39.810046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcdc05-ddc1-4c9a-8eaf-1e942a11df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:39.881694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-64dcdc05-ddc1-4c9a-8eaf-1e942a11df85 None None] MFA Rules not processed for user `215c643b0f074070a3ad68ec2459e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:39.883706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-64dcdc05-ddc1-4c9a-8eaf-1e942a11df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:39.898008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:39.899679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:40.017993 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 813/1647] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:39 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:40.020333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b051ec-6b9a-4274-9716-0f193bd61959 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:40.029868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:40.030896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:40.031382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:40.031831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] PATH_INFO: `/v3/projects/36fa703b5a7d4098ae0f4f7daadbeffb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:40.033682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=36fa703b5a7d4098ae0f4f7daadbeffb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:40.035916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: WARNING py.warnings [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:07:40.035916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: warnings.warn(msg) Aug 29 19:07:40.035916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: Aug 29 19:07:40.037795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:40.047782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:40.177290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36fa703b5a7d4098ae0f4f7daadbeffb', 'request_id': 'req-bc8067fb-685f-4f3c-a766-a297f639f6ab'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:40.198814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36fa703b5a7d4098ae0f4f7daadbeffb', 'request_id': 'req-bc8067fb-685f-4f3c-a766-a297f639f6ab'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:40.208495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36fa703b5a7d4098ae0f4f7daadbeffb', 'request_id': 'req-bc8067fb-685f-4f3c-a766-a297f639f6ab'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:40.227998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] The token cache is being invalidate because project 36fa703b5a7d4098ae0f4f7daadbeffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:07:40.228819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:40.229656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bc8067fb-685f-4f3c-a766-a297f639f6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:40.237182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46c155-e86b-4dfd-bb5a-1ce3d4d3d187 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:40.237643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46c155-e86b-4dfd-bb5a-1ce3d4d3d187 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:40.239267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46c155-e86b-4dfd-bb5a-1ce3d4d3d187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:40.246807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 835/1648] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:07:39 2022] DELETE /identity/v3/projects/36fa703b5a7d4098ae0f4f7daadbeffb => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:07:40.281356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1d46c155-e86b-4dfd-bb5a-1ce3d4d3d187 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:40.282940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1d46c155-e86b-4dfd-bb5a-1ce3d4d3d187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:40.469459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:40.495634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 814/1649] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:40.506111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6715d6c-30d1-4d39-86b4-5a0dc849785f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:40.506401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6715d6c-30d1-4d39-86b4-5a0dc849785f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:40.506682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6715d6c-30d1-4d39-86b4-5a0dc849785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:40.545784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f6715d6c-30d1-4d39-86b4-5a0dc849785f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:40.547381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f6715d6c-30d1-4d39-86b4-5a0dc849785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:40.574751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 836/1650] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:40.575639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46c155-e86b-4dfd-bb5a-1ce3d4d3d187 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:40.591138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46c155-e86b-4dfd-bb5a-1ce3d4d3d187 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:40.603507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:40.604091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:40.606672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:40.606959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:40.609352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:40.612315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:40.616133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:40.673019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 815/1651] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:40.678280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6715d6c-30d1-4d39-86b4-5a0dc849785f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:40.690203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6715d6c-30d1-4d39-86b4-5a0dc849785f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:40.708964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:40.709609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:40.709890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:40.710163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:40.710855 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:40.712848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:40.779722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 837/1652] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:40 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:40.784031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:40.803633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3943d-b7f9-49de-8ff7-7522f0954b8c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:40.901453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:40.902060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:40.902174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:40.902553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:40.903571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:40.905651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:40.932477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 816/1653] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:40 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:40.936710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:40.951451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e459-1452-4d95-92cf-d10d826ffe66 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:40.957037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:40.958157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:40.958646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:40.959064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:40.959995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:40.962021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:40.980860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:40.991387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 838/1654] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:40.994647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0ed89-5a1e-4028-abfc-2d73b071db5c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:41.000853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff9965-70ee-4ae9-8a5d-e15b832859c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:41.003221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff9965-70ee-4ae9-8a5d-e15b832859c0 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.003221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff9965-70ee-4ae9-8a5d-e15b832859c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.003221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff9965-70ee-4ae9-8a5d-e15b832859c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.003221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff9965-70ee-4ae9-8a5d-e15b832859c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:41.005402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff9965-70ee-4ae9-8a5d-e15b832859c0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:41.023751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:41.035269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 817/1655] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:07:41.039183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a41701-2c2c-44b9-9d27-4b1f68880009 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:41.045545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:41.046322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.046744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.047162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] PATH_INFO: `/v3/projects/8433291bce3443098967a4b0ebf04313/users/066bf4053a78497fa4f0b80567872596/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.068864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8433291bce3443098967a4b0ebf04313, user_id=066bf4053a78497fa4f0b80567872596, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:41.072276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:41.093946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 839/1656] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:41 2022] PUT /identity/v3/projects/8433291bce3443098967a4b0ebf04313/users/066bf4053a78497fa4f0b80567872596/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:41.105732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194ea0-1191-46a3-bd98-51f5fad3d0f4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.106024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194ea0-1191-46a3-bd98-51f5fad3d0f4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.106301 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194ea0-1191-46a3-bd98-51f5fad3d0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.170171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c9194ea0-1191-46a3-bd98-51f5fad3d0f4 None None] MFA Rules not processed for user `066bf4053a78497fa4f0b80567872596`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:41.172672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c9194ea0-1191-46a3-bd98-51f5fad3d0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:41.283692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 818/1657] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:41.289127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:41.407307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e77e25-f639-4bfa-a5f0-024f81acdbf0 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:41.416431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c2da6-27eb-4395-9e5f-84e09f14d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:41.417378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2da6-27eb-4395-9e5f-84e09f14d3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.417940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2da6-27eb-4395-9e5f-84e09f14d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.417940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2da6-27eb-4395-9e5f-84e09f14d3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.418991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c2da6-27eb-4395-9e5f-84e09f14d3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:41.420940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c2da6-27eb-4395-9e5f-84e09f14d3f6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:41.462507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 840/1658] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:41 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:07:41.466542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194ea0-1191-46a3-bd98-51f5fad3d0f4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:41.482203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194ea0-1191-46a3-bd98-51f5fad3d0f4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:41.568206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c448006-4a0b-4ec4-8dec-caa4b325b165 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.568594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c448006-4a0b-4ec4-8dec-caa4b325b165 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.569031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c448006-4a0b-4ec4-8dec-caa4b325b165 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.571937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 841/1659] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:07:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:07:41.572703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:41.591974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:41.593174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.593464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.593745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.594533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:41.601086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:41.617071 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.617596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.618416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.664415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 819/1660] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:41 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:41.667672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:41.693896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:41.694105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b4ffb-9399-4c73-96a3-758884085b91 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:41.702049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:41.705653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:41.706347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.706631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.707628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.708328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:41.709976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:41.737847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 820/1661] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:41.739347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:41.750504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:41.764691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2507fb-beb3-4eda-99b0-42bfd03d0f05 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:41.774746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:41.775463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:41.775740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:41.776037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] PATH_INFO: `/v3/projects/eeb08de236e04ad1b5fd3ebf0f1e48e1/users/13f8e10c8cc24eadb7150d438d273575/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:41.813392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb08de236e04ad1b5fd3ebf0f1e48e1, user_id=13f8e10c8cc24eadb7150d438d273575, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:41.816264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:41.857586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 821/1662] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:41 2022] PUT /identity/v3/projects/eeb08de236e04ad1b5fd3ebf0f1e48e1/users/13f8e10c8cc24eadb7150d438d273575/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:41.861641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:41.898398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 842/1663] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:07:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:41.917123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:41.921097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:41.965104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c9c2e-513d-4f80-9a81-18247264232a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:42.042701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e8582-ddf5-4578-898f-43da4c8f31a3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:42.061906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:42.062556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:42.062987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:42.063948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:42.064555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:42.067055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:42.137032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 822/1664] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:07:42.137509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:42.165868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:42.166564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:42.166841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:42.167837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:42.183429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903d40b-d0e3-4720-807a-119e1b623f7f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:42.206651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:42.209062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:42.212991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc71ed9-5154-420e-b357-1c69de2bf220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:42.214352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc71ed9-5154-420e-b357-1c69de2bf220 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:42.214985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc71ed9-5154-420e-b357-1c69de2bf220 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:42.215792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc71ed9-5154-420e-b357-1c69de2bf220 admin admin] PATH_INFO: `/v3/projects/eeb08de236e04ad1b5fd3ebf0f1e48e1/users/13f8e10c8cc24eadb7150d438d273575/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:42.218820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc71ed9-5154-420e-b357-1c69de2bf220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb08de236e04ad1b5fd3ebf0f1e48e1, user_id=13f8e10c8cc24eadb7150d438d273575, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:42.227326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc71ed9-5154-420e-b357-1c69de2bf220 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:42.263841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 823/1665] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:42 2022] PUT /identity/v3/projects/eeb08de236e04ad1b5fd3ebf0f1e48e1/users/13f8e10c8cc24eadb7150d438d273575/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:42.265378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:42.265670 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:42.265949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:42.267864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 843/1666] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:42.320456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] MFA Rules not processed for user `13f8e10c8cc24eadb7150d438d273575`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:42.328592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:42.522761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 824/1667] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:42.530307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:42.532006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:42.696223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826626fd-7191-4bfb-a51c-8bd6fdd93c86 service placement] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:42.815534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:42.816224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:42.816610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:42.817913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:42.834352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:42.944382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:42.947152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:43.037465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 844/1668] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:43.976704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:43.979273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:44.035786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:44.179661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8885bb-d613-4b94-a00b-587a8641eca4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:44.192744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:44.193758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:44.195380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:44.195741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] PATH_INFO: `/v3/users/e452d88233244dc8ada7372e7a30ef79` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:44.200041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e452d88233244dc8ada7372e7a30ef79)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:44.201630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: WARNING py.warnings [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 19:07:44.201630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: warnings.warn(msg) Aug 29 19:07:44.201630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: Aug 29 19:07:44.207012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:44.401790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e452d88233244dc8ada7372e7a30ef79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:44.495754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e452d88233244dc8ada7372e7a30ef79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:44.496677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e452d88233244dc8ada7372e7a30ef79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:44.497469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e452d88233244dc8ada7372e7a30ef79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:44.531542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e452d88233244dc8ada7372e7a30ef79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:44.576890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 825/1669] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:07:43 2022] DELETE /identity/v3/users/e452d88233244dc8ada7372e7a30ef79 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:45.331183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:45.358900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fe0b1-30a0-4b92-b1ec-cbfb64dbe4b2 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:45.559964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:45.561739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:45.564240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:45.567386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] PATH_INFO: `/v3/projects/88c406ecbec9468487e33d84fd16afd2` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:45.577434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c406ecbec9468487e33d84fd16afd2)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:45.622754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:45.783440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c406ecbec9468487e33d84fd16afd2', 'request_id': 'req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:45.808551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c406ecbec9468487e33d84fd16afd2', 'request_id': 'req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:45.846387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c406ecbec9468487e33d84fd16afd2', 'request_id': 'req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:45.896840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] The token cache is being invalidate because project 88c406ecbec9468487e33d84fd16afd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:07:45.897176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:45.901858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:45.908834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 845/1670] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:07:45 2022] DELETE /identity/v3/projects/88c406ecbec9468487e33d84fd16afd2 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:45.912700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:45.916321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:45.945505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902c26c-3759-4352-9906-a4e16380bb23 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:46.078931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:46.079863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:46.080159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:46.080546 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] PATH_INFO: `/v3/users/215c643b0f074070a3ad68ec2459e44f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:46.084608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215c643b0f074070a3ad68ec2459e44f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:46.088138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:46.188514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:46.208922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215c643b0f074070a3ad68ec2459e44f'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:46.288908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215c643b0f074070a3ad68ec2459e44f'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:46.289724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215c643b0f074070a3ad68ec2459e44f'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:46.290424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215c643b0f074070a3ad68ec2459e44f'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:46.304943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55caf10-a438-4802-8851-81c5ab0d4021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215c643b0f074070a3ad68ec2459e44f'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:46.336341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 826/1671] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:07:45 2022] DELETE /identity/v3/users/215c643b0f074070a3ad68ec2459e44f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:47.554676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:47.557417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:47.610905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d09f1-3dc2-4429-a8a6-80e88f1d7bce admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:47.844030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:47.882899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:47.884765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:47.885272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:47.885745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] PATH_INFO: `/v3/projects/f00fed15633e43d8a7df880cf9a62213` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:47.889411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00fed15633e43d8a7df880cf9a62213)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:47.897589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:48.076223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00fed15633e43d8a7df880cf9a62213', 'request_id': 'req-a83879a1-f302-4f7e-a0e2-41f358b63701'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:48.093353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00fed15633e43d8a7df880cf9a62213', 'request_id': 'req-a83879a1-f302-4f7e-a0e2-41f358b63701'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:48.111689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00fed15633e43d8a7df880cf9a62213', 'request_id': 'req-a83879a1-f302-4f7e-a0e2-41f358b63701'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:48.146614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] The token cache is being invalidate because project f00fed15633e43d8a7df880cf9a62213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:07:48.147537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:48.148475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a83879a1-f302-4f7e-a0e2-41f358b63701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:07:48.169179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 846/1672] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:07:47 2022] DELETE /identity/v3/projects/f00fed15633e43d8a7df880cf9a62213 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:48.174045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a6e1-d564-4660-b603-0570e4934d95 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.174566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a6e1-d564-4660-b603-0570e4934d95 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.174654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a6e1-d564-4660-b603-0570e4934d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.235798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-eb72a6e1-d564-4660-b603-0570e4934d95 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:48.237265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-eb72a6e1-d564-4660-b603-0570e4934d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:48.264862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798206b-b803-4ef8-af3a-278d05d46bbd None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.265305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798206b-b803-4ef8-af3a-278d05d46bbd None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.265744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798206b-b803-4ef8-af3a-278d05d46bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.398891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2798206b-b803-4ef8-af3a-278d05d46bbd None None] MFA Rules not processed for user `73c4a7ab7fc7425389843798faa25372`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:48.403908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2798206b-b803-4ef8-af3a-278d05d46bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:48.424085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 827/1673] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:48.427817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1bb8-aeff-4e10-8949-184b8d48e4d9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.428777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1bb8-aeff-4e10-8949-184b8d48e4d9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.429817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1bb8-aeff-4e10-8949-184b8d48e4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.503015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-57eb1bb8-aeff-4e10-8949-184b8d48e4d9 None None] MFA Rules not processed for user `624a0a90866446439918bd65ce5ddc60`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:48.506394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-57eb1bb8-aeff-4e10-8949-184b8d48e4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:48.576841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 847/1674] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:48.579430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b46f-ee62-4881-aadb-d984bbcd7c3e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.580234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b46f-ee62-4881-aadb-d984bbcd7c3e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.580715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b46f-ee62-4881-aadb-d984bbcd7c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.628819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-daf9b46f-ee62-4881-aadb-d984bbcd7c3e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:48.634309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-daf9b46f-ee62-4881-aadb-d984bbcd7c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:48.661561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 848/1675] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:48.663276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10bafb-ffc6-442e-9b5c-bf402b464783 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.666042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10bafb-ffc6-442e-9b5c-bf402b464783 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.667878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10bafb-ffc6-442e-9b5c-bf402b464783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.705991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 828/1676] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:48.709691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1bb8-aeff-4e10-8949-184b8d48e4d9 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:48.738547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1e10bafb-ffc6-442e-9b5c-bf402b464783 None None] MFA Rules not processed for user `73c4a7ab7fc7425389843798faa25372`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:48.743163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1bb8-aeff-4e10-8949-184b8d48e4d9 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:48.743463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1e10bafb-ffc6-442e-9b5c-bf402b464783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:48.808951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdebf05-ef29-454d-b0df-66ef27cc81ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:48.809918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdebf05-ef29-454d-b0df-66ef27cc81ef admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.809918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdebf05-ef29-454d-b0df-66ef27cc81ef admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.810139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdebf05-ef29-454d-b0df-66ef27cc81ef admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.810809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdebf05-ef29-454d-b0df-66ef27cc81ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:07:48.811140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdebf05-ef29-454d-b0df-66ef27cc81ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:48.816813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 849/1677] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:48.820938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd2e07-3526-4a7f-8eb3-2a802a837300 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.820938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd2e07-3526-4a7f-8eb3-2a802a837300 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.820938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd2e07-3526-4a7f-8eb3-2a802a837300 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.822915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdebf05-ef29-454d-b0df-66ef27cc81ef admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:48.825859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 850/1678] 10.222.0.94 () {54 vars in 962 bytes} [Mon Aug 29 19:07:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:07:48.842007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cd329-8959-4ee3-a79d-fa8b2e69e096 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.842300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cd329-8959-4ee3-a79d-fa8b2e69e096 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.842733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cd329-8959-4ee3-a79d-fa8b2e69e096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.870476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 829/1679] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:07:48.877394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939028e4-3415-4964-b87a-b112ae6ddd5d None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.877792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939028e4-3415-4964-b87a-b112ae6ddd5d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.878081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939028e4-3415-4964-b87a-b112ae6ddd5d None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.879990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 830/1680] 10.222.0.94 () {54 vars in 962 bytes} [Mon Aug 29 19:07:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:07:48.881940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939028e4-3415-4964-b87a-b112ae6ddd5d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:48.915877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939028e4-3415-4964-b87a-b112ae6ddd5d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:48.934317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc0772-cce0-4a22-8e99-eaa1e67ffae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:48.934943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc0772-cce0-4a22-8e99-eaa1e67ffae6 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:48.939606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc0772-cce0-4a22-8e99-eaa1e67ffae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:48.939916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc0772-cce0-4a22-8e99-eaa1e67ffae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:48.940592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc0772-cce0-4a22-8e99-eaa1e67ffae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:48.948959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc0772-cce0-4a22-8e99-eaa1e67ffae6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:48.957839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-766cd329-8959-4ee3-a79d-fa8b2e69e096 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:48.966003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-766cd329-8959-4ee3-a79d-fa8b2e69e096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:49.067225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 831/1681] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:48 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:49.073160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340cb5b-cd22-4327-9005-f8105802ed50 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.079197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340cb5b-cd22-4327-9005-f8105802ed50 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.079651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340cb5b-cd22-4327-9005-f8105802ed50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.109856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:49.156149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-3340cb5b-cd22-4327-9005-f8105802ed50 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:49.157415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3340cb5b-cd22-4327-9005-f8105802ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:49.179563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:49.238586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 851/1682] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Aug 29 19:07:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:49.239677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cd329-8959-4ee3-a79d-fa8b2e69e096 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:49.283987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cd329-8959-4ee3-a79d-fa8b2e69e096 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:49.418621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 832/1683] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Aug 29 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:49.421302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340cb5b-cd22-4327-9005-f8105802ed50 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:49.449005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340cb5b-cd22-4327-9005-f8105802ed50 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:49.474061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:49.476625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.479349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.479680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.522648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:49.524192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.524627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.525096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.526067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:49.529288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:49.593873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 852/1684] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:49 2022] POST /identity/v3/users => generated 290 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:49.594803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:49.621960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:49.623895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d0eea-b710-442b-8457-b0b8ac6fe06a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:49.624159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:49.634040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:49.635254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.635663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.635863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.639884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 833/1685] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:49.641120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:49.665139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93678-a7ec-4379-9bac-880bc1c33f34 service nova] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:49.679485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:49.681456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.681899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.682454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.683410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:49.688275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:49.732675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 834/1686] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:49.738969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:49.748768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:49.750827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:49.760083 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc03a-fd3d-4fc0-a05b-1ad4ede59c42 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:49.770046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2443cb-75ff-4096-836d-882bed2ade27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:49.771908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2443cb-75ff-4096-836d-882bed2ade27 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.773132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2443cb-75ff-4096-836d-882bed2ade27 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.774971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2443cb-75ff-4096-836d-882bed2ade27 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.775863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2443cb-75ff-4096-836d-882bed2ade27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:49.778013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2443cb-75ff-4096-836d-882bed2ade27 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:49.778440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 853/1687] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:49.791154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 835/1688] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:49.795789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:49.811238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d0394-0d90-4f33-ad5c-a73203d12150 service nova] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:49.816854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:49.817698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.818178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.818605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] PATH_INFO: `/v3/projects/e540243ce71b4e7ea25773adf2e1cb65/users/85a22238ea034d63987896b9bb902b87/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.842936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e540243ce71b4e7ea25773adf2e1cb65, user_id=85a22238ea034d63987896b9bb902b87, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:49.846450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:49.867199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 854/1689] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:49 2022] PUT /identity/v3/projects/e540243ce71b4e7ea25773adf2e1cb65/users/85a22238ea034d63987896b9bb902b87/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:49.877327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737539a5-046c-4328-be4f-436268c02a69 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:49.877629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737539a5-046c-4328-be4f-436268c02a69 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:49.877978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737539a5-046c-4328-be4f-436268c02a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:49.947250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-737539a5-046c-4328-be4f-436268c02a69 None None] MFA Rules not processed for user `85a22238ea034d63987896b9bb902b87`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:49.948915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-737539a5-046c-4328-be4f-436268c02a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:50.166841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:50.169072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:50.214221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 836/1690] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:07:50.220471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737539a5-046c-4328-be4f-436268c02a69 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:50.270631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:50.311495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:50.392830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d734da-22d0-4a2a-a324-9afbacd56a56 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:50.409147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c3275-fedf-4400-9749-383ecd76fd6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:50.411832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3275-fedf-4400-9749-383ecd76fd6b service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:50.413188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3275-fedf-4400-9749-383ecd76fd6b service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:50.413787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3275-fedf-4400-9749-383ecd76fd6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:50.418659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9b2c3275-fedf-4400-9749-383ecd76fd6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:50.471099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737539a5-046c-4328-be4f-436268c02a69 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:50.489195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98142a6-5a11-4fd7-ae5f-094c68471bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:50.490060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98142a6-5a11-4fd7-ae5f-094c68471bee admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:50.491907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98142a6-5a11-4fd7-ae5f-094c68471bee admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:50.493254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98142a6-5a11-4fd7-ae5f-094c68471bee admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:50.494407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98142a6-5a11-4fd7-ae5f-094c68471bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:50.502147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98142a6-5a11-4fd7-ae5f-094c68471bee admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:50.591961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 837/1691] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:07:50 2022] POST /identity/v3/projects => generated 379 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:50.595085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a1a5-a4e6-4971-a608-44b24edc60ed None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:50.596802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a1a5-a4e6-4971-a608-44b24edc60ed None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:50.599620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a1a5-a4e6-4971-a608-44b24edc60ed None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:50.601674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 838/1692] 10.222.0.94 () {54 vars in 912 bytes} [Mon Aug 29 19:07:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:07:50.602612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a1a5-a4e6-4971-a608-44b24edc60ed None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:50.619294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c3275-fedf-4400-9749-383ecd76fd6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:50.620921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c3275-fedf-4400-9749-383ecd76fd6b service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:50.647987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 855/1693] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:07:50.650306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:50.650601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:50.650878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:50.704415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:50.705685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:50.884307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a1a5-a4e6-4971-a608-44b24edc60ed None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:50.901470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:50.902112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:50.902399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:50.902673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:50.903607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:50.916435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:50.948887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 856/1694] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:07:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:50.985423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:50.987306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:51.016549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 839/1695] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:07:50 2022] POST /identity/v3/users => generated 296 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:07:51.023364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:51.079598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7dc62-05b8-4cbd-be1c-09efb95c5e1f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:51.095961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:51.096655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:51.098879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:51.099457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:51.100770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:51.105346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:51.141797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 840/1696] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:51.144055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:51.200317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4b374-12dc-4282-b07b-a6127e013f0e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:51.222228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:51.231033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:51.231386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:51.231664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] PATH_INFO: `/v3/projects/21ac098a8bf9455fbc65f5fb3e2b63d2/users/1a1b43384aa84d75ba91b7030e2cca03/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:51.259540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb8bc-bc36-4f1c-b344-1113e33b9486 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:51.272569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:51.278783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:51.284816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:51.286208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:51.299569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ac098a8bf9455fbc65f5fb3e2b63d2, user_id=1a1b43384aa84d75ba91b7030e2cca03, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:51.314769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:51.329511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:51.346415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:51.383538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 841/1697] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:51 2022] PUT /identity/v3/projects/21ac098a8bf9455fbc65f5fb3e2b63d2/users/1a1b43384aa84d75ba91b7030e2cca03/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:51.387813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:51.392791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:51.423460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 857/1698] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:51.576389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:51.626239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37407b0-62a0-4eb0-9ae1-ae074dff9595 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:51.650663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884637b-6de3-4cf2-a393-a89cc5a86cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:51.654611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884637b-6de3-4cf2-a393-a89cc5a86cdf admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:51.656402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884637b-6de3-4cf2-a393-a89cc5a86cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:51.657647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884637b-6de3-4cf2-a393-a89cc5a86cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:51.661941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884637b-6de3-4cf2-a393-a89cc5a86cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:51.667992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884637b-6de3-4cf2-a393-a89cc5a86cdf admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:51.699184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 842/1699] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:07:51.703323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:51.723982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040a28d-6240-4a14-81d5-378e1dcf2b99 service glance] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:51.740279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:51.741105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:51.741388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:51.741663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] PATH_INFO: `/v3/projects/21ac098a8bf9455fbc65f5fb3e2b63d2/users/1a1b43384aa84d75ba91b7030e2cca03/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:51.743793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ac098a8bf9455fbc65f5fb3e2b63d2, user_id=1a1b43384aa84d75ba91b7030e2cca03, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:51.746481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:51.777306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 858/1700] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:07:51 2022] PUT /identity/v3/projects/21ac098a8bf9455fbc65f5fb3e2b63d2/users/1a1b43384aa84d75ba91b7030e2cca03/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:07:51.791380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a175a1-dfd2-4a11-a0b9-eb09acc4e898 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:51.794055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a175a1-dfd2-4a11-a0b9-eb09acc4e898 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:51.794606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a175a1-dfd2-4a11-a0b9-eb09acc4e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:51.872929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a6a175a1-dfd2-4a11-a0b9-eb09acc4e898 None None] MFA Rules not processed for user `1a1b43384aa84d75ba91b7030e2cca03`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:51.879637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a6a175a1-dfd2-4a11-a0b9-eb09acc4e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:52.103004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 843/1701] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:52.108734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:52.253199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512e027-415a-4848-8a71-32531d2a4712 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:52.259468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdba38-13e8-48bb-8251-275f19389b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:52.260111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdba38-13e8-48bb-8251-275f19389b63 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:52.260394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdba38-13e8-48bb-8251-275f19389b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:52.260671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdba38-13e8-48bb-8251-275f19389b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:52.288301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdba38-13e8-48bb-8251-275f19389b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:52.289772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdba38-13e8-48bb-8251-275f19389b63 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:52.311503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 859/1702] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:52.419980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee9b9f-05bd-4a6f-af3d-69f022a2c7f2 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:52.420432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee9b9f-05bd-4a6f-af3d-69f022a2c7f2 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:52.420517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee9b9f-05bd-4a6f-af3d-69f022a2c7f2 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:52.422396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 844/1703] 10.222.0.94 () {54 vars in 912 bytes} [Mon Aug 29 19:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:07:52.434088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be815b-504f-4c4e-8a0c-05d6e8c18d62 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:52.434473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be815b-504f-4c4e-8a0c-05d6e8c18d62 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:52.434730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be815b-504f-4c4e-8a0c-05d6e8c18d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:52.508567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e7ea-743e-4220-adc3-ad0d290bafbc None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:52.508946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e7ea-743e-4220-adc3-ad0d290bafbc None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:52.509694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e7ea-743e-4220-adc3-ad0d290bafbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:52.667262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ec03e7ea-743e-4220-adc3-ad0d290bafbc None None] MFA Rules not processed for user `ff8ce50a95b742a48500e634db50fb36`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:52.668424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ec03e7ea-743e-4220-adc3-ad0d290bafbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:52.673505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c2be815b-504f-4c4e-8a0c-05d6e8c18d62 None None] MFA Rules not processed for user `70c6682b92694484b2fa2710314f547f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:52.676589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c2be815b-504f-4c4e-8a0c-05d6e8c18d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:52.883839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 845/1704] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:52.884745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03e7ea-743e-4220-adc3-ad0d290bafbc None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:52.915743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03e7ea-743e-4220-adc3-ad0d290bafbc None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:52.919763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:52.920337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:52.920638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:52.921092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:52.990845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 860/1705] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:07:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:52.992124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be815b-504f-4c4e-8a0c-05d6e8c18d62 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:53.085473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:53.088657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:53.111237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:53.127582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 846/1706] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:53.130597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:53.135309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:53.176259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1f21e-c472-45d5-b7c8-e5472544755c service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:53.181064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be815b-504f-4c4e-8a0c-05d6e8c18d62 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:53.192189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775e9fd-e0d9-4c90-aefa-56d88d216f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:53.192613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237d010-5d9c-4754-bbc2-ccab7d765163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:53.193410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d010-5d9c-4754-bbc2-ccab7d765163 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:53.193668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775e9fd-e0d9-4c90-aefa-56d88d216f13 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:53.193771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d010-5d9c-4754-bbc2-ccab7d765163 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:53.193946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d010-5d9c-4754-bbc2-ccab7d765163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:53.194730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775e9fd-e0d9-4c90-aefa-56d88d216f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:53.197196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775e9fd-e0d9-4c90-aefa-56d88d216f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:53.217357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775e9fd-e0d9-4c90-aefa-56d88d216f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:53.220259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237d010-5d9c-4754-bbc2-ccab7d765163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:53.226624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237d010-5d9c-4754-bbc2-ccab7d765163 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:53.231426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775e9fd-e0d9-4c90-aefa-56d88d216f13 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:53.252230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 847/1707] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:53.260341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 861/1708] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:53.562732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b2061-6a32-4f1d-9671-030530ec6cb7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:53.563568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b2061-6a32-4f1d-9671-030530ec6cb7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:53.563860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b2061-6a32-4f1d-9671-030530ec6cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:53.640304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-039b2061-6a32-4f1d-9671-030530ec6cb7 None None] MFA Rules not processed for user `cc580d348f154c4488a5f84640539655`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:53.645549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-039b2061-6a32-4f1d-9671-030530ec6cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:53.668828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a86e1-c707-46c2-8eac-9fb987785dfa None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:53.669660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a86e1-c707-46c2-8eac-9fb987785dfa None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:53.670937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a86e1-c707-46c2-8eac-9fb987785dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:53.724362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:53.760018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-708a86e1-c707-46c2-8eac-9fb987785dfa None None] MFA Rules not processed for user `ac0bc9e010d044a5a2147010cbdb4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:53.761935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-708a86e1-c707-46c2-8eac-9fb987785dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:53.841180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 848/1709] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:53.855609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243828-b5b8-4ce7-a7d0-77d71770234a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:53.855901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243828-b5b8-4ce7-a7d0-77d71770234a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:53.857108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243828-b5b8-4ce7-a7d0-77d71770234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:53.902322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7f243828-b5b8-4ce7-a7d0-77d71770234a None None] MFA Rules not processed for user `cc580d348f154c4488a5f84640539655`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:53.904241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7f243828-b5b8-4ce7-a7d0-77d71770234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:53.931734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 849/1710] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:53.939503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243828-b5b8-4ce7-a7d0-77d71770234a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:53.953569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 862/1711] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:53.973683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243828-b5b8-4ce7-a7d0-77d71770234a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:53.980255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150797d7-2e5f-4ae5-afd3-3545754c878a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:53.981085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150797d7-2e5f-4ae5-afd3-3545754c878a service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:53.981514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150797d7-2e5f-4ae5-afd3-3545754c878a service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:53.982101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150797d7-2e5f-4ae5-afd3-3545754c878a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:53.992952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150797d7-2e5f-4ae5-afd3-3545754c878a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:53.995504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150797d7-2e5f-4ae5-afd3-3545754c878a service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:54.007943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 850/1712] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:54.088801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9aeeb-1b6f-4ae0-8171-0ca452c48b88 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:54.089327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9aeeb-1b6f-4ae0-8171-0ca452c48b88 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:54.089470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9aeeb-1b6f-4ae0-8171-0ca452c48b88 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:54.090919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 863/1713] 10.222.0.94 () {54 vars in 912 bytes} [Mon Aug 29 19:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:07:54.100929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8f1d-681e-4e61-8dbf-7a934bf16967 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:54.101222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8f1d-681e-4e61-8dbf-7a934bf16967 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:54.101505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8f1d-681e-4e61-8dbf-7a934bf16967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:54.129399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-84cf8f1d-681e-4e61-8dbf-7a934bf16967 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:54.131120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-84cf8f1d-681e-4e61-8dbf-7a934bf16967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:54.155357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 851/1714] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:07:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:07:54.164451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9aeeb-1b6f-4ae0-8171-0ca452c48b88 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:54.167504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-14b9aeeb-1b6f-4ae0-8171-0ca452c48b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:54.271979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9aeeb-1b6f-4ae0-8171-0ca452c48b88 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:54.278026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:54.279447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:54.279959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:54.280212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:54.298198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:54.301956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:54.318651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 864/1715] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:54.423543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8f1d-681e-4e61-8dbf-7a934bf16967 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:54.446804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8f1d-681e-4e61-8dbf-7a934bf16967 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:54.470361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dd57e-974a-4482-8b21-ca9b3448c591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:54.474043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dd57e-974a-4482-8b21-ca9b3448c591 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:54.474738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dd57e-974a-4482-8b21-ca9b3448c591 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:54.476018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dd57e-974a-4482-8b21-ca9b3448c591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:54.517923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dd57e-974a-4482-8b21-ca9b3448c591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:54.525027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dd57e-974a-4482-8b21-ca9b3448c591 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:54.565481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 852/1716] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:54.651507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:54.653135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:54.865898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea19593-f37c-4888-9c26-78075e5f39c2 service glance] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:54.887583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dbf73-2a05-4ad5-8a3e-f1a6422d3a19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:54.888525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbf73-2a05-4ad5-8a3e-f1a6422d3a19 service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:54.888811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbf73-2a05-4ad5-8a3e-f1a6422d3a19 service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:54.889091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbf73-2a05-4ad5-8a3e-f1a6422d3a19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:54.892407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7f1dbf73-2a05-4ad5-8a3e-f1a6422d3a19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:54.939055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dbf73-2a05-4ad5-8a3e-f1a6422d3a19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:54.943970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dbf73-2a05-4ad5-8a3e-f1a6422d3a19 service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:54.969821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 865/1717] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:55.180511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57d857-3ff5-4b16-bca9-9b3a0bd1ee56 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:55.181742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57d857-3ff5-4b16-bca9-9b3a0bd1ee56 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:55.181742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57d857-3ff5-4b16-bca9-9b3a0bd1ee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:55.228169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-da57d857-3ff5-4b16-bca9-9b3a0bd1ee56 None None] MFA Rules not processed for user `73c4a7ab7fc7425389843798faa25372`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:55.233424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-da57d857-3ff5-4b16-bca9-9b3a0bd1ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:55.376667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6954-7e44-4e47-8001-777853fcc270 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:55.377496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6954-7e44-4e47-8001-777853fcc270 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:55.377957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b6954-7e44-4e47-8001-777853fcc270 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:55.380290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 866/1718] 10.222.0.94 () {54 vars in 912 bytes} [Mon Aug 29 19:07:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:07:55.387954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cfaf4-1af0-4a65-a59f-05c2c5142d3a None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:55.388817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cfaf4-1af0-4a65-a59f-05c2c5142d3a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:55.389287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cfaf4-1af0-4a65-a59f-05c2c5142d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:55.427850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f75cfaf4-1af0-4a65-a59f-05c2c5142d3a None None] MFA Rules not processed for user `70c6682b92694484b2fa2710314f547f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:55.435828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f75cfaf4-1af0-4a65-a59f-05c2c5142d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:55.445803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 867/1719] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:07:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:55.462418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cfaf4-1af0-4a65-a59f-05c2c5142d3a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:07:55.482779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cfaf4-1af0-4a65-a59f-05c2c5142d3a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:07:55.496797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 853/1720] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:07:55.504764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e7e05-4023-4e5e-903d-2ea1b2c67de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:07:55.511585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e7e05-4023-4e5e-903d-2ea1b2c67de8 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:55.512080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e7e05-4023-4e5e-903d-2ea1b2c67de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:55.512477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e7e05-4023-4e5e-903d-2ea1b2c67de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:55.530764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e7e05-4023-4e5e-903d-2ea1b2c67de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:07:55.532517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e7e05-4023-4e5e-903d-2ea1b2c67de8 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:07:55.592115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 868/1721] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:07:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:07:59.353152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431826-a7e4-4877-8cd9-28d55220695f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:07:59.354056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431826-a7e4-4877-8cd9-28d55220695f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:07:59.354538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431826-a7e4-4877-8cd9-28d55220695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:07:59.417861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d5431826-a7e4-4877-8cd9-28d55220695f None None] MFA Rules not processed for user `624a0a90866446439918bd65ce5ddc60`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:07:59.422824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d5431826-a7e4-4877-8cd9-28d55220695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:07:59.546177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:07:59.618684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 854/1722] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:00.001920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe23593-05cf-4daa-a8a3-bf4bed1f9f52 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:00.002535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe23593-05cf-4daa-a8a3-bf4bed1f9f52 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:00.002825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe23593-05cf-4daa-a8a3-bf4bed1f9f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:00.061495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-dbe23593-05cf-4daa-a8a3-bf4bed1f9f52 None None] MFA Rules not processed for user `ff8ce50a95b742a48500e634db50fb36`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:00.064385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dbe23593-05cf-4daa-a8a3-bf4bed1f9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:00.079257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 869/1723] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:07:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:00.548900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:00.549404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:00.549488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:00.569710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] MFA Rules not processed for user `ac0bc9e010d044a5a2147010cbdb4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:00.571047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:00.579269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 855/1724] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:04.935300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad13b-ba8e-45bb-8364-ca43c7b8ece2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:04.936372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad13b-ba8e-45bb-8364-ca43c7b8ece2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:04.936830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad13b-ba8e-45bb-8364-ca43c7b8ece2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:04.950926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:05.096889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-06bad13b-ba8e-45bb-8364-ca43c7b8ece2 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:05.100356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-06bad13b-ba8e-45bb-8364-ca43c7b8ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:05.331223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 870/1725] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:08:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:05.570140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:05.570140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:05.828087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef0fe0-44a2-463e-aaf9-e74c9fcf8d1a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:05.844509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d0bac-02e1-4c00-a363-b30477b00017 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:05.845884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0bac-02e1-4c00-a363-b30477b00017 service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:05.847324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0bac-02e1-4c00-a363-b30477b00017 service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:05.847324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0bac-02e1-4c00-a363-b30477b00017 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:05.867119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d0bac-02e1-4c00-a363-b30477b00017 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:05.867261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d0bac-02e1-4c00-a363-b30477b00017 service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:05.887812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 856/1726] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:08:10.867303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bad13b-ba8e-45bb-8364-ca43c7b8ece2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:10.943469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bad13b-ba8e-45bb-8364-ca43c7b8ece2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:10.948418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:10.949353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:10.949844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:10.950202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] PATH_INFO: `/v3/users/85a22238ea034d63987896b9bb902b87` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:10.951748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a22238ea034d63987896b9bb902b87)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:10.954222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:11.020101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a22238ea034d63987896b9bb902b87'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.048401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a22238ea034d63987896b9bb902b87'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.049313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a22238ea034d63987896b9bb902b87'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.064349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a22238ea034d63987896b9bb902b87'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.076092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:11.083952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a22238ea034d63987896b9bb902b87'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.095688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 871/1727] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:10 2022] DELETE /identity/v3/users/85a22238ea034d63987896b9bb902b87 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:11.100976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:11.101995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:11.102198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:11.141583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:11.143598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:11.223642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 857/1728] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:11.234546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:11.236140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:11.323981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef96a83-d905-41f4-b2e2-b926fa567445 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:11.332078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:11.333167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:11.333616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:11.334119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:11.352619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:11.354348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:11.370000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 872/1729] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:11.789553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:11.791014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:11.804289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e61fa3-bc91-4c73-b77f-479bed1125d6 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:11.809768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:11.810620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:11.811017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:11.811376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] PATH_INFO: `/v3/projects/e540243ce71b4e7ea25773adf2e1cb65` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:11.812587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e540243ce71b4e7ea25773adf2e1cb65)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:11.814590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:11.875875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:11.879831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e540243ce71b4e7ea25773adf2e1cb65', 'request_id': 'req-1ce988b2-1c28-44da-bf33-f9bf2584b964'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.887177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e540243ce71b4e7ea25773adf2e1cb65', 'request_id': 'req-1ce988b2-1c28-44da-bf33-f9bf2584b964'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.894955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e540243ce71b4e7ea25773adf2e1cb65', 'request_id': 'req-1ce988b2-1c28-44da-bf33-f9bf2584b964'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.906588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] The token cache is being invalidate because project e540243ce71b4e7ea25773adf2e1cb65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:11.906879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.907595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:11.909147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 858/1730] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:11 2022] DELETE /identity/v3/projects/e540243ce71b4e7ea25773adf2e1cb65 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:11.914573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:11.924784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15780b-0818-4a15-ab43-9ee0539c8329 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:11.990282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:11.991034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:11.991458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:11.991806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] PATH_INFO: `/v3/users/1a1b43384aa84d75ba91b7030e2cca03` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:11.993029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1b43384aa84d75ba91b7030e2cca03)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:11.995156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:12.049149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1b43384aa84d75ba91b7030e2cca03'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.058038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1b43384aa84d75ba91b7030e2cca03'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.058678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1b43384aa84d75ba91b7030e2cca03'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.071703 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1b43384aa84d75ba91b7030e2cca03'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.078551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-12710a38-9c46-4cfe-8817-19d4b4595333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1b43384aa84d75ba91b7030e2cca03'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.086117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 873/1731] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:11 2022] DELETE /identity/v3/users/1a1b43384aa84d75ba91b7030e2cca03 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:12.420522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:12.425214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:12.458886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce988b2-1c28-44da-bf33-f9bf2584b964 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:12.593688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:12.594453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:12.594864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:12.595245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] PATH_INFO: `/v3/projects/21ac098a8bf9455fbc65f5fb3e2b63d2` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:12.596515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ac098a8bf9455fbc65f5fb3e2b63d2)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:12.606895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:12.687576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ac098a8bf9455fbc65f5fb3e2b63d2', 'request_id': 'req-9553a33a-1d04-44a7-a783-ae3eaea39f79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.697179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ac098a8bf9455fbc65f5fb3e2b63d2', 'request_id': 'req-9553a33a-1d04-44a7-a783-ae3eaea39f79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.728785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ac098a8bf9455fbc65f5fb3e2b63d2', 'request_id': 'req-9553a33a-1d04-44a7-a783-ae3eaea39f79'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.759679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] The token cache is being invalidate because project 21ac098a8bf9455fbc65f5fb3e2b63d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:12.760112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.764428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-9553a33a-1d04-44a7-a783-ae3eaea39f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:12.774264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 859/1732] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:12 2022] DELETE /identity/v3/projects/21ac098a8bf9455fbc65f5fb3e2b63d2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:12.780280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57d17e-911e-4100-b256-bd082274bb44 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:12.780748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57d17e-911e-4100-b256-bd082274bb44 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:12.781208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57d17e-911e-4100-b256-bd082274bb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:12.864883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8a57d17e-911e-4100-b256-bd082274bb44 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:12.867969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8a57d17e-911e-4100-b256-bd082274bb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:12.884583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:12.983001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 874/1733] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:12.991162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90cbdf-14ed-414a-afa7-08da6aaf6722 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:12.991439 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90cbdf-14ed-414a-afa7-08da6aaf6722 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:12.991705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90cbdf-14ed-414a-afa7-08da6aaf6722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.020115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cf90cbdf-14ed-414a-afa7-08da6aaf6722 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:13.021598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cf90cbdf-14ed-414a-afa7-08da6aaf6722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:13.034174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 860/1734] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:13.036497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57d17e-911e-4100-b256-bd082274bb44 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.061541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57d17e-911e-4100-b256-bd082274bb44 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.066498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.067345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.067655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.067964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.068749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:13.069074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.070687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.085762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 875/1735] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:13.090580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90cbdf-14ed-414a-afa7-08da6aaf6722 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.101449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90cbdf-14ed-414a-afa7-08da6aaf6722 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.106611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.107357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.107711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.108063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.108784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.111147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.160509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 861/1736] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:13 2022] POST /identity/v3/projects => generated 355 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:13.161661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.174332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e0b2c-89a0-4231-bea7-30a6fb8d001e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.274820 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.275351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.275655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.275955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.276662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.278806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.304924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 876/1737] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:13 2022] POST /identity/v3/users => generated 278 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:13.308761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.328891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c4cd-408c-4fd0-8c64-0477beb3f7f2 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.335367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.336055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.336279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.336550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.337475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.339122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.351581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 862/1738] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:13.356098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.375723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33b12-2788-4318-96da-15699c4895b6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.390060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079b667-b6af-4581-9bd5-2ea42f7373a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.390781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079b667-b6af-4581-9bd5-2ea42f7373a5 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.391171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079b667-b6af-4581-9bd5-2ea42f7373a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.391519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079b667-b6af-4581-9bd5-2ea42f7373a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.392246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079b667-b6af-4581-9bd5-2ea42f7373a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.394104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079b667-b6af-4581-9bd5-2ea42f7373a5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.422471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 877/1739] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:13.427154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.437970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db009ad-fde7-4db7-b6d8-0377672322d4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.443706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.444426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.444725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.444984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] PATH_INFO: `/v3/projects/712c6b9874704942b3b8a217181bbea4/users/7bf75940bcb24e94b83fbe5fe99b1775/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.461966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712c6b9874704942b3b8a217181bbea4, user_id=7bf75940bcb24e94b83fbe5fe99b1775, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.464702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.473514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:13.481033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 863/1740] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:13 2022] PUT /identity/v3/projects/712c6b9874704942b3b8a217181bbea4/users/7bf75940bcb24e94b83fbe5fe99b1775/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:13.492019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e6612-8af9-4785-94e7-ae157093881b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.492304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e6612-8af9-4785-94e7-ae157093881b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.492583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e6612-8af9-4785-94e7-ae157093881b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.525500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-6a1e6612-8af9-4785-94e7-ae157093881b None None] MFA Rules not processed for user `7bf75940bcb24e94b83fbe5fe99b1775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:13.526745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6a1e6612-8af9-4785-94e7-ae157093881b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:13.604944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 878/1741] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:13.611628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.731535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4752d1-76dc-4ba6-a0dc-83b91128ded6 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.739061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.739895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.740204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.740494 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.741153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.743231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.774906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 864/1742] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:13 2022] POST /identity/v3/projects => generated 355 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:13.778613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e6612-8af9-4785-94e7-ae157093881b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.795440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e6612-8af9-4785-94e7-ae157093881b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.817994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:13.882923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.883699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.884203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.884492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.885292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.887362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.915014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 879/1743] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:13 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:13.916201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.929855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c2f34-d2f0-486b-b356-b45ac0187689 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.935381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.936382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.936728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.937149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:13.937926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:13.939602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:13.956829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:13.961282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 865/1744] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:13.984236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dd3a9-f7bc-4e58-a316-a1b17afb38c3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:13.990182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:13.990893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:13.991264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:13.991550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] PATH_INFO: `/v3/projects/ef98fed9c49c4c81ae982d257a1ff6c1/users/1e94cd75ad3a4d68acb1153c0f073e2c/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:14.032806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef98fed9c49c4c81ae982d257a1ff6c1, user_id=1e94cd75ad3a4d68acb1153c0f073e2c, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:14.036321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:14.052885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 880/1745] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:13 2022] PUT /identity/v3/projects/ef98fed9c49c4c81ae982d257a1ff6c1/users/1e94cd75ad3a4d68acb1153c0f073e2c/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:14.073270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:14.087699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf361e-f1d9-49c2-a64a-92373ec30412 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:14.219959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dee0d-91ea-4209-a8b4-60663c4e8a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:14.220683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dee0d-91ea-4209-a8b4-60663c4e8a48 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:14.221246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dee0d-91ea-4209-a8b4-60663c4e8a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:14.221949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dee0d-91ea-4209-a8b4-60663c4e8a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:14.222592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030dee0d-91ea-4209-a8b4-60663c4e8a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:14.224919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030dee0d-91ea-4209-a8b4-60663c4e8a48 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:14.237952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 866/1746] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:14.243044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:14.258917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d94865-5ab3-4b5d-a36d-e5c5f3268d82 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:14.269348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:14.271657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:14.271657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:14.271657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] PATH_INFO: `/v3/projects/ef98fed9c49c4c81ae982d257a1ff6c1/users/1e94cd75ad3a4d68acb1153c0f073e2c/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:14.274993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef98fed9c49c4c81ae982d257a1ff6c1, user_id=1e94cd75ad3a4d68acb1153c0f073e2c, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:14.278015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:14.311123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 881/1747] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:14 2022] PUT /identity/v3/projects/ef98fed9c49c4c81ae982d257a1ff6c1/users/1e94cd75ad3a4d68acb1153c0f073e2c/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:14.326587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7f07-6e19-4e23-ac7e-5e66decb252a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:14.327251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7f07-6e19-4e23-ac7e-5e66decb252a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:14.327731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7f07-6e19-4e23-ac7e-5e66decb252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:14.388404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c5ea7f07-6e19-4e23-ac7e-5e66decb252a None None] MFA Rules not processed for user `1e94cd75ad3a4d68acb1153c0f073e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:14.391128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c5ea7f07-6e19-4e23-ac7e-5e66decb252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:14.537361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:14.553767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 867/1748] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:14.568298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:14.576919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:14.660908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fd8d8-8151-4622-8405-bee79c196764 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:14.664713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6fdc6-7b2b-44dd-ba8d-93e3cd7bd285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:14.665336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6fdc6-7b2b-44dd-ba8d-93e3cd7bd285 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:14.665612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6fdc6-7b2b-44dd-ba8d-93e3cd7bd285 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:14.665882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6fdc6-7b2b-44dd-ba8d-93e3cd7bd285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:14.682867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6fdc6-7b2b-44dd-ba8d-93e3cd7bd285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:14.685973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6fdc6-7b2b-44dd-ba8d-93e3cd7bd285 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:14.698109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 882/1749] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:14.799164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea7f07-6e19-4e23-ac7e-5e66decb252a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:14.813827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea7f07-6e19-4e23-ac7e-5e66decb252a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:14.819765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:14.823754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:14.823964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:14.824447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:14.949189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:14.951447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:14.990848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 868/1750] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:17.671870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08458fd7-0325-47c8-a163-91cf95aa12b4 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:17.672416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08458fd7-0325-47c8-a163-91cf95aa12b4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:17.672416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08458fd7-0325-47c8-a163-91cf95aa12b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:17.736116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-08458fd7-0325-47c8-a163-91cf95aa12b4 None None] MFA Rules not processed for user `70c6682b92694484b2fa2710314f547f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:17.738356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-08458fd7-0325-47c8-a163-91cf95aa12b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:17.907547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 883/1751] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Aug 29 19:08:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:17.964158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:17.966183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:17.988120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ac748-c479-40f5-be9a-f28c9d626925 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:17.992188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de7f0a-1308-4a1f-8d87-01e03f1865fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:17.992880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de7f0a-1308-4a1f-8d87-01e03f1865fc service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:17.993256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de7f0a-1308-4a1f-8d87-01e03f1865fc service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:17.993600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de7f0a-1308-4a1f-8d87-01e03f1865fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:18.001532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de7f0a-1308-4a1f-8d87-01e03f1865fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:18.003043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de7f0a-1308-4a1f-8d87-01e03f1865fc service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:18.012478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 869/1752] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:18.883434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7aaf1-f367-4c68-97b6-697432bc7430 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:18.883958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7aaf1-f367-4c68-97b6-697432bc7430 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:18.884287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7aaf1-f367-4c68-97b6-697432bc7430 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:18.885751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 884/1753] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:08:18.892704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:18.893279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:18.893770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:18.927790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:18.929001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:18.997034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 870/1754] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:08:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:19.010913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7aaf1-f367-4c68-97b6-697432bc7430 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:19.012701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d6f7aaf1-f367-4c68-97b6-697432bc7430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:19.031512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7aaf1-f367-4c68-97b6-697432bc7430 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:19.037987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d1991-149d-4665-aaac-11a8a7c83097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:19.038655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d1991-149d-4665-aaac-11a8a7c83097 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:19.038941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d1991-149d-4665-aaac-11a8a7c83097 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:19.039315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d1991-149d-4665-aaac-11a8a7c83097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:19.049036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d1991-149d-4665-aaac-11a8a7c83097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:19.050629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d1991-149d-4665-aaac-11a8a7c83097 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:19.062090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 885/1755] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:23.155670 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:23.157797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:23.333178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d39dd0-26ee-44c8-952a-6b55519fcb3c None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:23.344278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:23.345086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:23.345368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:23.345639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] PATH_INFO: `/v3/users/066bf4053a78497fa4f0b80567872596` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:23.347267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=066bf4053a78497fa4f0b80567872596)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:23.350674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:23.375338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:23.439746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066bf4053a78497fa4f0b80567872596'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:23.468912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066bf4053a78497fa4f0b80567872596'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:23.470123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066bf4053a78497fa4f0b80567872596'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:23.471461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066bf4053a78497fa4f0b80567872596'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:23.483550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066bf4053a78497fa4f0b80567872596'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:23.508248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 871/1756] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:23 2022] DELETE /identity/v3/users/066bf4053a78497fa4f0b80567872596 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:23.512384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61046ffc-5365-4134-950e-99dd965a3994 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:23.513234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61046ffc-5365-4134-950e-99dd965a3994 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:23.513718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61046ffc-5365-4134-950e-99dd965a3994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:23.525367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:23.551233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-61046ffc-5365-4134-950e-99dd965a3994 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:23.553717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-61046ffc-5365-4134-950e-99dd965a3994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:23.642812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 886/1757] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:23.672522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:23.674601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:23.749858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10121f6-7867-4b27-b2a9-a5e42d0701b2 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:23.755491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8c19b-ed60-482e-83eb-9d8706db6f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:23.756170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c19b-ed60-482e-83eb-9d8706db6f2c service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:23.756659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c19b-ed60-482e-83eb-9d8706db6f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:23.757018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c19b-ed60-482e-83eb-9d8706db6f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:23.769205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8c19b-ed60-482e-83eb-9d8706db6f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:23.770854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8c19b-ed60-482e-83eb-9d8706db6f2c service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:23.786085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 872/1758] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:24.518598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b49e4d-68d6-4537-bc82-356e45c50374 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:24.519617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b49e4d-68d6-4537-bc82-356e45c50374 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:24.520190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b49e4d-68d6-4537-bc82-356e45c50374 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:24.522568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 887/1759] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:08:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:08:24.542400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:24.542919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:24.542992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:24.589168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:24.590901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:24.719540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 873/1760] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:08:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:25.162948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b49e4d-68d6-4537-bc82-356e45c50374 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:25.165903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e9b49e4d-68d6-4537-bc82-356e45c50374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:25.204981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b49e4d-68d6-4537-bc82-356e45c50374 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:25.215780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:25.216420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:25.219659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:25.219976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:25.262290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:25.263956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:25.295455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 888/1761] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:25.907365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:25.908944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:25.934838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bd36f-2e6f-41d3-a712-487170ee69cf None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:25.947729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:25.948574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:25.949227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:25.949701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] PATH_INFO: `/v3/projects/8433291bce3443098967a4b0ebf04313` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:25.954838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8433291bce3443098967a4b0ebf04313)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:25.959623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:25.992282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:26.037772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8433291bce3443098967a4b0ebf04313', 'request_id': 'req-8413daf3-5111-42cf-9f57-9ba4b03a371c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.047028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8433291bce3443098967a4b0ebf04313', 'request_id': 'req-8413daf3-5111-42cf-9f57-9ba4b03a371c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.056950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8433291bce3443098967a4b0ebf04313', 'request_id': 'req-8413daf3-5111-42cf-9f57-9ba4b03a371c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.084866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] The token cache is being invalidate because project 8433291bce3443098967a4b0ebf04313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:26.087174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.088651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.091171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 874/1762] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:25 2022] DELETE /identity/v3/projects/8433291bce3443098967a4b0ebf04313 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:26.095250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:26.113831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d7bd-796d-4fe7-8c13-b0e33cc4aaca service nova] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:26.272357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:26.273359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:26.274009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:26.276015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] PATH_INFO: `/v3/users/13f8e10c8cc24eadb7150d438d273575` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:26.279786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f8e10c8cc24eadb7150d438d273575)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:26.282373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:26.493578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f8e10c8cc24eadb7150d438d273575'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.510609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f8e10c8cc24eadb7150d438d273575'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.511492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f8e10c8cc24eadb7150d438d273575'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.532759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f8e10c8cc24eadb7150d438d273575'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.543371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8a53297b-4bbb-4ba0-9180-8199c0fe975d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f8e10c8cc24eadb7150d438d273575'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:26.551101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 889/1763] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:26 2022] DELETE /identity/v3/users/13f8e10c8cc24eadb7150d438d273575 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:27.522113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:27.524613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:27.547551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413daf3-5111-42cf-9f57-9ba4b03a371c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:27.628932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:27.631610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:27.632035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:27.632518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] PATH_INFO: `/v3/projects/eeb08de236e04ad1b5fd3ebf0f1e48e1` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:27.635800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb08de236e04ad1b5fd3ebf0f1e48e1)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:27.641851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:27.744141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb08de236e04ad1b5fd3ebf0f1e48e1', 'request_id': 'req-6dde523e-8574-4889-a2d7-f1d5ae52d825'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.752198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb08de236e04ad1b5fd3ebf0f1e48e1', 'request_id': 'req-6dde523e-8574-4889-a2d7-f1d5ae52d825'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.764796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb08de236e04ad1b5fd3ebf0f1e48e1', 'request_id': 'req-6dde523e-8574-4889-a2d7-f1d5ae52d825'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.789865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] The token cache is being invalidate because project eeb08de236e04ad1b5fd3ebf0f1e48e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:27.790509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.791417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6dde523e-8574-4889-a2d7-f1d5ae52d825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:27.793175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 875/1764] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:27 2022] DELETE /identity/v3/projects/eeb08de236e04ad1b5fd3ebf0f1e48e1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:27.798831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10012047-2fea-43bb-9c77-8026edaa42e8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:27.799270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10012047-2fea-43bb-9c77-8026edaa42e8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:27.799433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10012047-2fea-43bb-9c77-8026edaa42e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:27.823258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-10012047-2fea-43bb-9c77-8026edaa42e8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:27.824766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-10012047-2fea-43bb-9c77-8026edaa42e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:27.838164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:27.900352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 890/1765] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:27.908388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df37214-4375-4924-afbc-4c4f082ea59c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:27.908675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df37214-4375-4924-afbc-4c4f082ea59c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:27.908975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df37214-4375-4924-afbc-4c4f082ea59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:27.957210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1df37214-4375-4924-afbc-4c4f082ea59c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:27.959901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1df37214-4375-4924-afbc-4c4f082ea59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:27.971751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 876/1766] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:27.976585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10012047-2fea-43bb-9c77-8026edaa42e8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:27.988450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10012047-2fea-43bb-9c77-8026edaa42e8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:27.994925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:27.995698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:27.995983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:27.996253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:27.996954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:27.997255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:27.999027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.012657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 891/1767] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:28.017259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df37214-4375-4924-afbc-4c4f082ea59c None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.030991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df37214-4375-4924-afbc-4c4f082ea59c None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.036686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.037596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.037882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.038156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.039453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.041762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.110790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 877/1768] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:28.115708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.138549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5288d-ee55-45f0-a62c-19d80fbe3d6b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.248997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.249577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.249866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.250145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.250798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.254489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.301559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 892/1769] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:28.306229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.322611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70729c-d69e-4c94-96b8-299f89f3cda9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.329219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.330321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.330616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.330891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.332093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.334231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.338487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:28.355645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 878/1770] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:28.361158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.401483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bafba-ea8d-4ec2-a796-dfa8f18048b9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.407838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448e29-c165-4bdd-8527-4741477467b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.409194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448e29-c165-4bdd-8527-4741477467b6 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.409654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448e29-c165-4bdd-8527-4741477467b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.410111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448e29-c165-4bdd-8527-4741477467b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.411630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448e29-c165-4bdd-8527-4741477467b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.417868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448e29-c165-4bdd-8527-4741477467b6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.431720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 893/1771] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:28.436142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.453396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a4b07-6502-471f-af14-cf2a4a37c8ee admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.460148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.461099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.461503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.461881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] PATH_INFO: `/v3/projects/30bf22b7c47445ec9ec0c803c81bc197/users/8a27d810690d4a469969441be8be2607/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.483238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30bf22b7c47445ec9ec0c803c81bc197, user_id=8a27d810690d4a469969441be8be2607, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.486129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.518074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 879/1772] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:28 2022] PUT /identity/v3/projects/30bf22b7c47445ec9ec0c803c81bc197/users/8a27d810690d4a469969441be8be2607/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:28.519835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab459-be55-400f-a6a2-4e8b4f74ce30 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.520140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab459-be55-400f-a6a2-4e8b4f74ce30 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.520400 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab459-be55-400f-a6a2-4e8b4f74ce30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.550995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b7dab459-be55-400f-a6a2-4e8b4f74ce30 None None] MFA Rules not processed for user `8a27d810690d4a469969441be8be2607`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:28.552299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b7dab459-be55-400f-a6a2-4e8b4f74ce30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:28.626376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 894/1773] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:28.630724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.714765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d695c-ae20-4127-8a8d-10f7a6044269 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.719659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.720283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.720571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.720852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.721492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.723445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.752461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 880/1774] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:28.756625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab459-be55-400f-a6a2-4e8b4f74ce30 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.768455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab459-be55-400f-a6a2-4e8b4f74ce30 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.774301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:28.857522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.857522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.857522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.857522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.858469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.858469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.895159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 895/1775] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:28 2022] POST /identity/v3/users => generated 303 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:28.897920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.908917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1050c8-8d7b-4569-a522-cff50c6e078f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.913889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.914376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.914663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.914961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.915712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.919554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.940502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 881/1776] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:28.941803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:28.953027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec452a7-8fa2-4eb4-abb7-98c5150dd0eb admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:28.957741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:28.958404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:28.958681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:28.958959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] PATH_INFO: `/v3/projects/f6abe61ee1f34e83a96f0fa610488ccb/users/af63601242ac4e80b9e32c567ae2cbb5/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:28.977078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6abe61ee1f34e83a96f0fa610488ccb, user_id=af63601242ac4e80b9e32c567ae2cbb5, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:28.980055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:28.998776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 896/1777] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:28 2022] PUT /identity/v3/projects/f6abe61ee1f34e83a96f0fa610488ccb/users/af63601242ac4e80b9e32c567ae2cbb5/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:29.002794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:29.013500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15df47-08d9-4374-935d-dc291af0c89f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:29.083812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654d29d-5247-4176-bffd-7c82097e5c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:29.085172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654d29d-5247-4176-bffd-7c82097e5c08 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:29.086550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654d29d-5247-4176-bffd-7c82097e5c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:29.086856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654d29d-5247-4176-bffd-7c82097e5c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:29.087617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654d29d-5247-4176-bffd-7c82097e5c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:29.089345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654d29d-5247-4176-bffd-7c82097e5c08 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:29.104886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 882/1778] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:29.106169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:29.118068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8400d-e665-4224-9a48-1516943bc296 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:29.130668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:29.131818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:29.132187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:29.132778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] PATH_INFO: `/v3/projects/f6abe61ee1f34e83a96f0fa610488ccb/users/af63601242ac4e80b9e32c567ae2cbb5/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:29.135831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6abe61ee1f34e83a96f0fa610488ccb, user_id=af63601242ac4e80b9e32c567ae2cbb5, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:29.139472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:29.153124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 897/1779] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:29 2022] PUT /identity/v3/projects/f6abe61ee1f34e83a96f0fa610488ccb/users/af63601242ac4e80b9e32c567ae2cbb5/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:29.166500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:29.166896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:29.167375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:29.204609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] MFA Rules not processed for user `af63601242ac4e80b9e32c567ae2cbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:29.206420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:29.256781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:29.300590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 883/1780] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:29.312698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:29.314736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:29.385382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81915b3-0b55-4fd2-bed4-e51c3228e011 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:29.392584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e603b-bb60-4b1d-8530-3f95191ac6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:29.393258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e603b-bb60-4b1d-8530-3f95191ac6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:29.393539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e603b-bb60-4b1d-8530-3f95191ac6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:29.393812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e603b-bb60-4b1d-8530-3f95191ac6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:29.404267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e603b-bb60-4b1d-8530-3f95191ac6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:29.405974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e603b-bb60-4b1d-8530-3f95191ac6d7 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:29.417328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 898/1781] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:29 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:29.480533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:29.482026 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:29.494951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8834f81-ff2a-496e-84a6-7f60232d77b0 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:29.498653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4bdfd-6406-4c69-bb4c-b86416918333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:29.499284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4bdfd-6406-4c69-bb4c-b86416918333 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:29.499562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4bdfd-6406-4c69-bb4c-b86416918333 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:29.499832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4bdfd-6406-4c69-bb4c-b86416918333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:29.559580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4bdfd-6406-4c69-bb4c-b86416918333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:29.561049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4bdfd-6406-4c69-bb4c-b86416918333 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:29.570280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 884/1782] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:33.530454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff78b3-1d57-4edf-b88d-3be43110404b None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.530971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff78b3-1d57-4edf-b88d-3be43110404b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.531048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff78b3-1d57-4edf-b88d-3be43110404b None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.532397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 899/1783] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:08:33.546576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e74bd-c574-454a-9f9e-99ae0dd6981e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.547119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e74bd-c574-454a-9f9e-99ae0dd6981e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.547218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e74bd-c574-454a-9f9e-99ae0dd6981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.567575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bc3e74bd-c574-454a-9f9e-99ae0dd6981e None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:33.569244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bc3e74bd-c574-454a-9f9e-99ae0dd6981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:33.651292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 885/1784] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:08:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:33.725584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff78b3-1d57-4edf-b88d-3be43110404b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:33.727531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7cff78b3-1d57-4edf-b88d-3be43110404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:33.750461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff78b3-1d57-4edf-b88d-3be43110404b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:33.766490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d34adf-94fb-4a7e-a187-f1886d6de573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:33.766931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d34adf-94fb-4a7e-a187-f1886d6de573 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:33.767253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d34adf-94fb-4a7e-a187-f1886d6de573 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:33.767608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d34adf-94fb-4a7e-a187-f1886d6de573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:33.777967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d34adf-94fb-4a7e-a187-f1886d6de573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:33.779989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d34adf-94fb-4a7e-a187-f1886d6de573 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:33.797518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 900/1785] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:34.301721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e74bd-c574-454a-9f9e-99ae0dd6981e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:34.385744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e74bd-c574-454a-9f9e-99ae0dd6981e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:34.396126 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:34.397763 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:34.398734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:34.399112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] PATH_INFO: `/v3/users/8a27d810690d4a469969441be8be2607` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:34.400831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a27d810690d4a469969441be8be2607)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:34.405338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:34.408463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:34.477464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a27d810690d4a469969441be8be2607'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:34.490560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a27d810690d4a469969441be8be2607'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:34.491319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a27d810690d4a469969441be8be2607'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:34.492808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a27d810690d4a469969441be8be2607'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:34.500645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a27d810690d4a469969441be8be2607'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:34.519662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 886/1786] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:34 2022] DELETE /identity/v3/users/8a27d810690d4a469969441be8be2607 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:34.526223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:34.526700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:34.527209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:34.552887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:34.555160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:34.604991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:34.631911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 901/1787] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:34.641696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:34.643180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:34.706842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e439-ba3c-46f1-9f05-73ce151f640e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:34.710948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:34.711836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:34.712121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:34.712394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:34.722748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:34.724332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:34.735360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 887/1788] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:08:35.016835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:35.018186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:35.032257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5de00-5e54-4dc6-9163-12fb10b55d84 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:35.064986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:35.064986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:35.065231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:35.065875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] PATH_INFO: `/v3/projects/30bf22b7c47445ec9ec0c803c81bc197` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:35.066940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] RBAC: Authorizing `identity:delete_project(project_id=30bf22b7c47445ec9ec0c803c81bc197)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:35.070286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:35.139565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30bf22b7c47445ec9ec0c803c81bc197', 'request_id': 'req-476139f0-476c-48e9-b362-b8e6a576d24a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.150708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30bf22b7c47445ec9ec0c803c81bc197', 'request_id': 'req-476139f0-476c-48e9-b362-b8e6a576d24a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.158590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30bf22b7c47445ec9ec0c803c81bc197', 'request_id': 'req-476139f0-476c-48e9-b362-b8e6a576d24a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.181377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] The token cache is being invalidate because project 30bf22b7c47445ec9ec0c803c81bc197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:35.181748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.182491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.187195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 902/1789] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:35 2022] DELETE /identity/v3/projects/30bf22b7c47445ec9ec0c803c81bc197 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:35.188512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:35.204932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3924-0b8e-426a-b956-214154dd914e service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:35.372143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:35.372809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:35.373093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:35.373371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] PATH_INFO: `/v3/users/af63601242ac4e80b9e32c567ae2cbb5` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:35.375755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af63601242ac4e80b9e32c567ae2cbb5)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:35.377725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:35.415224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:35.481329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af63601242ac4e80b9e32c567ae2cbb5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.509761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af63601242ac4e80b9e32c567ae2cbb5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.511406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af63601242ac4e80b9e32c567ae2cbb5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.512661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af63601242ac4e80b9e32c567ae2cbb5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.547780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-66868817-fecb-4acb-9983-c4caa64a7216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af63601242ac4e80b9e32c567ae2cbb5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:35.572299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 888/1790] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:35 2022] DELETE /identity/v3/users/af63601242ac4e80b9e32c567ae2cbb5 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:36.185949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:36.189550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:36.216266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476139f0-476c-48e9-b362-b8e6a576d24a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:36.337461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:36.338819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:36.339459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:36.339886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] PATH_INFO: `/v3/users/3cf9bdab54db49609ed37f819ce0ceb1` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:36.341287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf9bdab54db49609ed37f819ce0ceb1)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:36.343628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:36.441696 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf9bdab54db49609ed37f819ce0ceb1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.456797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf9bdab54db49609ed37f819ce0ceb1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.457611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf9bdab54db49609ed37f819ce0ceb1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.486001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf9bdab54db49609ed37f819ce0ceb1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.489170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:36.498589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf9bdab54db49609ed37f819ce0ceb1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.507680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 903/1791] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:36 2022] DELETE /identity/v3/users/3cf9bdab54db49609ed37f819ce0ceb1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:36.512297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:36.512879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:36.512961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:36.536021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:36.537572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:36.613832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 889/1792] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:36.624868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:36.626711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:36.697680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ae895-9c58-4d0a-a25b-11dd2c282bd1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:36.701442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49d58f-a23a-4cbf-9f93-9e19625352c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:36.702055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49d58f-a23a-4cbf-9f93-9e19625352c5 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:36.702790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49d58f-a23a-4cbf-9f93-9e19625352c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:36.702790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49d58f-a23a-4cbf-9f93-9e19625352c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:36.713121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49d58f-a23a-4cbf-9f93-9e19625352c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:36.714538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49d58f-a23a-4cbf-9f93-9e19625352c5 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:36.725011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 904/1793] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:36.780534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:36.782411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:36.812782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014606d-859c-42f0-8091-a56661f4d061 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:36.822564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:36.823216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:36.823493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:36.823786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] PATH_INFO: `/v3/projects/f6abe61ee1f34e83a96f0fa610488ccb` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:36.825686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6abe61ee1f34e83a96f0fa610488ccb)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:36.839427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:36.938014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6abe61ee1f34e83a96f0fa610488ccb', 'request_id': 'req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.958248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6abe61ee1f34e83a96f0fa610488ccb', 'request_id': 'req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.972023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6abe61ee1f34e83a96f0fa610488ccb', 'request_id': 'req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.988799 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] The token cache is being invalidate because project f6abe61ee1f34e83a96f0fa610488ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:36.989196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.990147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:36.996032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 890/1794] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:36 2022] DELETE /identity/v3/projects/f6abe61ee1f34e83a96f0fa610488ccb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:36.997612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea5e8-b29b-4a70-acf4-95109cfc4236 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:36.997965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea5e8-b29b-4a70-acf4-95109cfc4236 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:36.998771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cea5e8-b29b-4a70-acf4-95109cfc4236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.037546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a5cea5e8-b29b-4a70-acf4-95109cfc4236 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:37.041254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a5cea5e8-b29b-4a70-acf4-95109cfc4236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:37.102093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:37.104154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:37.158722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:37.202756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 905/1795] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:37.210584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdc0d6-3bb2-471d-97b6-bf3da0087248 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.211723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdc0d6-3bb2-471d-97b6-bf3da0087248 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.212446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdc0d6-3bb2-471d-97b6-bf3da0087248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.260963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-68fdc0d6-3bb2-471d-97b6-bf3da0087248 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:37.282232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-68fdc0d6-3bb2-471d-97b6-bf3da0087248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:37.317491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 906/1796] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:37.318737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdc0d6-3bb2-471d-97b6-bf3da0087248 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:37.332201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ab6e1-56ef-42e2-93b4-808b3d94c712 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:37.343104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdc0d6-3bb2-471d-97b6-bf3da0087248 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:37.361233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:37.362053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.362597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.364603 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.366069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:37.367484 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:37.368080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:37.368401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.368691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.370942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] PATH_INFO: `/v3/projects/48587976e5aa4379ac6844050b072405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.372720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:37.376852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] RBAC: Authorizing `identity:delete_project(project_id=48587976e5aa4379ac6844050b072405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:37.381083 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:37.407917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 907/1797] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:37.412964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:37.442091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5831101-75ec-49dd-86f7-371f5ffc0482 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:37.450120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:37.451445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.451993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.452549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.453542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:37.456674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:37.497695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48587976e5aa4379ac6844050b072405', 'request_id': 'req-e801fa2c-7665-412d-9859-558bd15f171e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.522277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48587976e5aa4379ac6844050b072405', 'request_id': 'req-e801fa2c-7665-412d-9859-558bd15f171e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.531688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 908/1798] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:37 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:37.535767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:37.553231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48587976e5aa4379ac6844050b072405', 'request_id': 'req-e801fa2c-7665-412d-9859-558bd15f171e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.581401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] The token cache is being invalidate because project 48587976e5aa4379ac6844050b072405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:37.581990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.582864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.587201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 891/1799] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:37 2022] DELETE /identity/v3/projects/48587976e5aa4379ac6844050b072405 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:37.590786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:37.593732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:37.614333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:37.631452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e098d1-f472-4023-8674-1a442c27e93c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:37.640736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:37.641160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.642499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.642499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.642955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:37.645921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:37.682930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 909/1800] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:37 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:37.684742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:37.686457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:37.702541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0a967-7888-4803-9cfe-bc9d003a2777 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:37.713464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:37.714455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.714844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.715293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] PATH_INFO: `/v3/users/7bf75940bcb24e94b83fbe5fe99b1775` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.717889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf75940bcb24e94b83fbe5fe99b1775)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:37.723581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:37.745948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801fa2c-7665-412d-9859-558bd15f171e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:37.753165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:37.753966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.754262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.754545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] PATH_INFO: `/v3/users/c7371bc7f7c8480e91f03987148a4b03` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.755857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7371bc7f7c8480e91f03987148a4b03)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:37.768653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:37.796888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf75940bcb24e94b83fbe5fe99b1775'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.811385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf75940bcb24e94b83fbe5fe99b1775'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.812436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf75940bcb24e94b83fbe5fe99b1775'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.837544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf75940bcb24e94b83fbe5fe99b1775'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.850474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7371bc7f7c8480e91f03987148a4b03'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.860998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf75940bcb24e94b83fbe5fe99b1775'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.873735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 910/1801] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:37 2022] DELETE /identity/v3/users/7bf75940bcb24e94b83fbe5fe99b1775 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:37.873735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:37.873981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:37.875916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:37.894611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7371bc7f7c8480e91f03987148a4b03'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.895678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7371bc7f7c8480e91f03987148a4b03'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.896556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7371bc7f7c8480e91f03987148a4b03'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.906146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f83784d-47c4-4a4f-a246-0826a95bbeea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7371bc7f7c8480e91f03987148a4b03'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:37.928523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 892/1802] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:37 2022] DELETE /identity/v3/users/c7371bc7f7c8480e91f03987148a4b03 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:37.931049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:37.931384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:37.932758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:37.955110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:37.956936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:37.957767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364300-df9e-4a8b-8155-c19b4987959d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.038306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 893/1803] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:38.053178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.055753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.058058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.059210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.059749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.060170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.061174 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.063343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.067733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:38.084112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 911/1804] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:38.093029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.095721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.112455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6779a-6324-46ab-af78-ea77a73b9877 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.117707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.118324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.118606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.118884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.120149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.121355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.132475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 912/1805] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:38.138969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.144906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb362d3-c2c2-4b56-8d00-e4e59f9aad8f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.150148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69938e3-5d54-46ed-b854-8c80beabaded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.150419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8e86d-3260-4342-a5c3-27d06ce52317 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.150939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69938e3-5d54-46ed-b854-8c80beabaded service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.151872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69938e3-5d54-46ed-b854-8c80beabaded service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.151872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69938e3-5d54-46ed-b854-8c80beabaded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.156021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.156750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.157107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.157459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] PATH_INFO: `/v3/projects/bcf4d7dd1c354906beb30b1d6d9e716d/users/c2997b1741154d76892b55cf1a250308/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.166789 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69938e3-5d54-46ed-b854-8c80beabaded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.168462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69938e3-5d54-46ed-b854-8c80beabaded service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.177319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf4d7dd1c354906beb30b1d6d9e716d, user_id=c2997b1741154d76892b55cf1a250308, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.180306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.184842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 894/1806] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:08:38.219117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 913/1807] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:38 2022] PUT /identity/v3/projects/bcf4d7dd1c354906beb30b1d6d9e716d/users/c2997b1741154d76892b55cf1a250308/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:38.227878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6e944-8816-40c5-b77e-17668431455e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.228404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6e944-8816-40c5-b77e-17668431455e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.228812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6e944-8816-40c5-b77e-17668431455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.305886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1ef6e944-8816-40c5-b77e-17668431455e None None] MFA Rules not processed for user `c2997b1741154d76892b55cf1a250308`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:38.307253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1ef6e944-8816-40c5-b77e-17668431455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.319625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.323021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.424839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 895/1808] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:38.430094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6e944-8816-40c5-b77e-17668431455e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.437811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56861c93-bc15-48b9-97ca-779727e41e4b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.443041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6e944-8816-40c5-b77e-17668431455e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.446430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.447140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.447427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.447701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] PATH_INFO: `/v3/projects/7fed10bc27eb4c4bbbb7526eeb36c89e` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.448896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fed10bc27eb4c4bbbb7526eeb36c89e)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.450100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.450806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.451450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.451836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.452223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.453106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.456276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.461280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:38.502956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 896/1809] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:38 2022] POST /identity/v3/projects => generated 389 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:38.510154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.553009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fed10bc27eb4c4bbbb7526eeb36c89e', 'request_id': 'req-d472f9ae-60d6-403f-89dd-472054900164'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.578510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fed10bc27eb4c4bbbb7526eeb36c89e', 'request_id': 'req-d472f9ae-60d6-403f-89dd-472054900164'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.585633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fed10bc27eb4c4bbbb7526eeb36c89e', 'request_id': 'req-d472f9ae-60d6-403f-89dd-472054900164'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.590525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259b83b-0562-4bdd-a831-48e2c21a5203 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.603931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] The token cache is being invalidate because project 7fed10bc27eb4c4bbbb7526eeb36c89e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:38.604232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.604886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d472f9ae-60d6-403f-89dd-472054900164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.606645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 914/1810] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:38 2022] DELETE /identity/v3/projects/7fed10bc27eb4c4bbbb7526eeb36c89e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:38.612617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.612911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.613192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.636271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:38.637510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.675648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.676270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.677042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.677042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.677442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.680175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.700689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:38.708105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 897/1811] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:38 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:38.712079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.714101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.720486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 915/1812] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:38.724008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.725503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.726098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148e37-3b91-4e41-b053-ab51695f0a38 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.731174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.731814 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.732098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.732381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] PATH_INFO: `/v3/projects/712c6b9874704942b3b8a217181bbea4` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.733726 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=712c6b9874704942b3b8a217181bbea4)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.736352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.737805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4022d9-399f-43ca-9df9-c13cec178441 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.743480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c057f-ee59-4b27-a71b-cb4f1f3d4392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.744144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c057f-ee59-4b27-a71b-cb4f1f3d4392 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.744423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c057f-ee59-4b27-a71b-cb4f1f3d4392 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.744693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c057f-ee59-4b27-a71b-cb4f1f3d4392 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.745329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c057f-ee59-4b27-a71b-cb4f1f3d4392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.746924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c057f-ee59-4b27-a71b-cb4f1f3d4392 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.759120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 916/1813] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:38.760982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.761874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.762461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.784783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:38.786094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.787187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712c6b9874704942b3b8a217181bbea4', 'request_id': 'req-73df20a7-c594-4d09-8cb9-6eecbbafb396'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.795286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712c6b9874704942b3b8a217181bbea4', 'request_id': 'req-73df20a7-c594-4d09-8cb9-6eecbbafb396'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.809328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712c6b9874704942b3b8a217181bbea4', 'request_id': 'req-73df20a7-c594-4d09-8cb9-6eecbbafb396'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.821956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] The token cache is being invalidate because project 712c6b9874704942b3b8a217181bbea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:38.822248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.823046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.824966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 898/1814] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:38 2022] DELETE /identity/v3/projects/712c6b9874704942b3b8a217181bbea4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:38.831613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.833031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.840146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:38.862147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 917/1815] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:38.863016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.868116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:38.880773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e5fe6-9248-4f23-8e52-add98b997cc8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.886295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.886936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.887250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.887655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] PATH_INFO: `/v3/users/1e94cd75ad3a4d68acb1153c0f073e2c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.888819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e94cd75ad3a4d68acb1153c0f073e2c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.890730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.911036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df20a7-c594-4d09-8cb9-6eecbbafb396 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:38.918352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:38.919384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:38.919858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:38.920299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] PATH_INFO: `/v3/projects/64c62a980ade402e843a31bedd35c693/users/e8bd092e54984669b587347b0de5a6c5/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:38.938537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c62a980ade402e843a31bedd35c693, user_id=e8bd092e54984669b587347b0de5a6c5, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:38.941288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:38.957260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e94cd75ad3a4d68acb1153c0f073e2c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.959568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 899/1816] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:38 2022] PUT /identity/v3/projects/64c62a980ade402e843a31bedd35c693/users/e8bd092e54984669b587347b0de5a6c5/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:38.960031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:38.972771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e94cd75ad3a4d68acb1153c0f073e2c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.973593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e94cd75ad3a4d68acb1153c0f073e2c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:38.992606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e94cd75ad3a4d68acb1153c0f073e2c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:39.003469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e94cd75ad3a4d68acb1153c0f073e2c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:39.013175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 918/1817] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:38 2022] DELETE /identity/v3/users/1e94cd75ad3a4d68acb1153c0f073e2c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:39.014265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.016228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:39.066902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4519b7-e85e-41c4-a7f6-cb5c85fb073d admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.157194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:39.167945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212541d-8d5d-494a-8344-8d288f4e8469 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.196600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.197248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.197526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.197800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.198459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.200114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.202324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.202959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.203267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.203543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.204255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:39.204515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.206115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.229288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 919/1818] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:39.230305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 900/1819] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:39.234380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.238390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.238548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:39.260629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24aa56-3fcc-4416-a1b6-5579eb32e273 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.263585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128683-5b1a-415a-9094-4e926baa3bc2 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.266615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11afc95-b13d-46b7-8051-76051dda469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.267510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11afc95-b13d-46b7-8051-76051dda469a admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.267754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11afc95-b13d-46b7-8051-76051dda469a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.268034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11afc95-b13d-46b7-8051-76051dda469a admin admin] PATH_INFO: `/v3/projects/64c62a980ade402e843a31bedd35c693/users/e8bd092e54984669b587347b0de5a6c5/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.269327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.270247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.270247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.270432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11afc95-b13d-46b7-8051-76051dda469a admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c62a980ade402e843a31bedd35c693, user_id=e8bd092e54984669b587347b0de5a6c5, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.270727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.271486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.274250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.275413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11afc95-b13d-46b7-8051-76051dda469a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.290956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 901/1820] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:39 2022] PUT /identity/v3/projects/64c62a980ade402e843a31bedd35c693/users/e8bd092e54984669b587347b0de5a6c5/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:39.303210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.303619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.303983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.334441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 920/1821] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:39 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:39.336210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.363638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] MFA Rules not processed for user `e8bd092e54984669b587347b0de5a6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:39.363638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:39.363638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:39.429118 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec48c-5ed3-4cf7-bcbb-c96e35cc5f18 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.434536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.434923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.435321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.435602 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.436176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.438129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.477409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 921/1822] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:39 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:39.479027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.487414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 902/1823] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:39 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:39.495167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532acf8-8c09-4537-b29d-82ee1fea253f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.500296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.500744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.501434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.501782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.502136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.502854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.505607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.506797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:39.519991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 922/1824] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:39.526227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.549746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309938ae-5004-4f8c-9a9b-d48d5030435d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.560901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.561617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.561992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.562342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.563089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.564777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.584316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 923/1825] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:39.591950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.604406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a06ccd-cf7e-4a9e-b517-3771fb2c7a76 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.611959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.612635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.612920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.613274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] PATH_INFO: `/v3/projects/74fd78a2177d4e83a644a528669825f7/users/590251bf628a48c88dccb2827af1a1d6/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.642698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74fd78a2177d4e83a644a528669825f7, user_id=590251bf628a48c88dccb2827af1a1d6, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.648485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.675416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 924/1826] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:39 2022] PUT /identity/v3/projects/74fd78a2177d4e83a644a528669825f7/users/590251bf628a48c88dccb2827af1a1d6/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:39.684688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a50cf-7ee5-4dc6-9814-a3c04f004776 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.685540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:39.687566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:39.769816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad7867-9171-4ff2-94df-b8d28c65ef5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.770638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad7867-9171-4ff2-94df-b8d28c65ef5c service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.771094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad7867-9171-4ff2-94df-b8d28c65ef5c service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.771379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad7867-9171-4ff2-94df-b8d28c65ef5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.800672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd6ddc-0332-4344-8031-8090f7c10d2e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:39.808897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:39.809800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.810182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.810540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] PATH_INFO: `/v3/projects/ef98fed9c49c4c81ae982d257a1ff6c1` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.811710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef98fed9c49c4c81ae982d257a1ff6c1)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.813849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.845056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:39.876335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad7867-9171-4ff2-94df-b8d28c65ef5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:39.878868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad7867-9171-4ff2-94df-b8d28c65ef5c service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:39.894977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 903/1827] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:39 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:39.897057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.897426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.897861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:39.933984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:39.937454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef98fed9c49c4c81ae982d257a1ff6c1', 'request_id': 'req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:39.953966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] MFA Rules not processed for user `590251bf628a48c88dccb2827af1a1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:39.955304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:39.956838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef98fed9c49c4c81ae982d257a1ff6c1', 'request_id': 'req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:39.967839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef98fed9c49c4c81ae982d257a1ff6c1', 'request_id': 'req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:39.985681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] The token cache is being invalidate because project ef98fed9c49c4c81ae982d257a1ff6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:39.986097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:39.986834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1bc6c0f9-a2c7-4299-bc9e-5f214fbc9dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:39.988659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 925/1828] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:39 2022] DELETE /identity/v3/projects/ef98fed9c49c4c81ae982d257a1ff6c1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:39.994928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:39.995347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:39.995679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.018979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:40.020292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.041732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 904/1829] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:40.047624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.049297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.098391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 926/1830] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:40.099199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.105049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.119526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dfcb8-ff96-493a-8b5d-ee941f60ed66 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.124175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.124983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.125278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.125557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] PATH_INFO: `/v3/users/c2997b1741154d76892b55cf1a250308` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.126846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2997b1741154d76892b55cf1a250308)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.128862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.133864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9409-5934-448b-afca-13760ba3b47e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.145381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd0add-021d-404e-8bb1-03987d4f9456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.146091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0add-021d-404e-8bb1-03987d4f9456 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.146563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0add-021d-404e-8bb1-03987d4f9456 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.146911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0add-021d-404e-8bb1-03987d4f9456 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.148616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd0add-021d-404e-8bb1-03987d4f9456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.150942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd0add-021d-404e-8bb1-03987d4f9456 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.187954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2997b1741154d76892b55cf1a250308'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:40.188169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 905/1831] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:40 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:40.189481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a36a-c352-4990-92eb-477b0531900d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.189773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a36a-c352-4990-92eb-477b0531900d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.190050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a36a-c352-4990-92eb-477b0531900d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.200695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2997b1741154d76892b55cf1a250308'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:40.201348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2997b1741154d76892b55cf1a250308'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:40.214155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ce59a36a-c352-4990-92eb-477b0531900d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:40.215590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ce59a36a-c352-4990-92eb-477b0531900d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.218961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2997b1741154d76892b55cf1a250308'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:40.227729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2997b1741154d76892b55cf1a250308'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:40.234456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 927/1832] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:40 2022] DELETE /identity/v3/users/c2997b1741154d76892b55cf1a250308 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:40.235388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.237281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.259167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:40.307186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 906/1833] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:40.308616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b22bb3-cbdc-4799-ba54-da9b6f165aac None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.308939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b22bb3-cbdc-4799-ba54-da9b6f165aac None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.309190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b22bb3-cbdc-4799-ba54-da9b6f165aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.328778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e3b22bb3-cbdc-4799-ba54-da9b6f165aac None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:40.328979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece6062-0913-43a9-91ad-df3682144434 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.329991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e3b22bb3-cbdc-4799-ba54-da9b6f165aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.333863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.335058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.336954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.337327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.337977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.339666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 907/1834] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:40.339977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.340475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b22bb3-cbdc-4799-ba54-da9b6f165aac None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.350177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b22bb3-cbdc-4799-ba54-da9b6f165aac None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.356389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.357160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.357457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.357737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.358495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:40.358877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.360513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.364865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 928/1835] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:40 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:40.365908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.368301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.374089 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 908/1836] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:40.375038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.381011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:40.389567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee39a7-74f1-492a-9f6e-e6134efc54d1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.397807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.398982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.399304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.399581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.400236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.401853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.415219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 909/1837] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:40.416049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.427404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c5889-d0d0-41b2-b5e0-bdcd0ab2b85a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.433155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.433764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.434045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.434324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.434961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.436886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.441967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fac25c-30fd-4f80-b3a0-348bc3352c73 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.450603 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.451320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.451608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.451886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.466406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.469469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.476404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 910/1838] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:40 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:40.476404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.508006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 929/1839] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:40.509612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.534147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48116871-7a9a-4591-9290-01e48d5b2fcc service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.553216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc9ae0-99c8-4d2c-8a10-763388735141 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.558861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.560415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.561042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.561781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] PATH_INFO: `/v3/projects/48617c39cab8430cac35ca731c4988a2/users/dc4a72a69aa24c1cb0ac56b456ec77f2/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.581685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=48617c39cab8430cac35ca731c4988a2, user_id=dc4a72a69aa24c1cb0ac56b456ec77f2, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.586335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.607421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 911/1840] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:40 2022] PUT /identity/v3/projects/48617c39cab8430cac35ca731c4988a2/users/dc4a72a69aa24c1cb0ac56b456ec77f2/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:40.611266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.636668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.637302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.637568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.637842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.638507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.640462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.671638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 930/1841] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:40 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:40.675642 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.689479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b66d4-18c1-4e83-a6f9-a3174c3e406c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.708964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780c3bc-89c9-4507-aa15-8ad180df9abe admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.718478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.719989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.720426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.721823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.722908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.725312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.729422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:40.739145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 912/1842] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:40.744750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.755610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60098c1-055b-4209-aa56-77d324c3fadb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.760625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e00a4-db8f-4af5-9b16-75ace62c3c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.761168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.761464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e00a4-db8f-4af5-9b16-75ace62c3c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.761788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.761889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e00a4-db8f-4af5-9b16-75ace62c3c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.762069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.762164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e00a4-db8f-4af5-9b16-75ace62c3c8b admin admin] PATH_INFO: `/v3/projects/48617c39cab8430cac35ca731c4988a2/users/dc4a72a69aa24c1cb0ac56b456ec77f2/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.762345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.763010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.764420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e00a4-db8f-4af5-9b16-75ace62c3c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48617c39cab8430cac35ca731c4988a2, user_id=dc4a72a69aa24c1cb0ac56b456ec77f2, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.764823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.767237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e00a4-db8f-4af5-9b16-75ace62c3c8b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.777474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 931/1843] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:40.781160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 913/1844] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:40 2022] PUT /identity/v3/projects/48617c39cab8430cac35ca731c4988a2/users/dc4a72a69aa24c1cb0ac56b456ec77f2/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:40.781987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.793076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70317c8d-6c05-4187-a7ae-c1c58173f203 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.793260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.793468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.793843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.852431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] MFA Rules not processed for user `dc4a72a69aa24c1cb0ac56b456ec77f2`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:40.854079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.873117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.873763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.874042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.874735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.875420 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.877505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:40.883332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:40.891820 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 932/1845] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:40.896852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.914927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68762c3e-b0f0-423b-8e95-30efaaad6104 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:40.935832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:40.936524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:40.936806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:40.937080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] PATH_INFO: `/v3/projects/f530303e20704b88b87100476573d181/users/67f6989c3c4d4fe3a2cc3be83a5f5f0c/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:40.946861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 914/1846] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:40.958404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:40.963658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:40.977779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f530303e20704b88b87100476573d181, user_id=67f6989c3c4d4fe3a2cc3be83a5f5f0c, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:40.982577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:41.007113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 933/1847] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:40 2022] PUT /identity/v3/projects/f530303e20704b88b87100476573d181/users/67f6989c3c4d4fe3a2cc3be83a5f5f0c/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:41.007113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:41.014917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.052853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce4144-7be4-49e3-9dcf-9a94bfd1aa65 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:41.119172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccdc3f-ffc1-4360-bf3f-b87263472d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:41.120231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccdc3f-ffc1-4360-bf3f-b87263472d62 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:41.120693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccdc3f-ffc1-4360-bf3f-b87263472d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:41.120977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccdc3f-ffc1-4360-bf3f-b87263472d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:41.137039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d8f72-c6a7-4b5b-8bb5-5d3380cf2863 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:41.145123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:41.146788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:41.147004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:41.147455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] PATH_INFO: `/v3/projects/bcf4d7dd1c354906beb30b1d6d9e716d` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:41.151164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcf4d7dd1c354906beb30b1d6d9e716d)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:41.153364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:41.214623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccdc3f-ffc1-4360-bf3f-b87263472d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:41.217098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccdc3f-ffc1-4360-bf3f-b87263472d62 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:41.230892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 915/1848] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:41.232189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:41.232479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:41.232817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:41.243456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:41.245474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcf4d7dd1c354906beb30b1d6d9e716d', 'request_id': 'req-977c113e-f856-45f9-ad34-c5e6afad8850'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.253946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:41.255220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.258386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcf4d7dd1c354906beb30b1d6d9e716d', 'request_id': 'req-977c113e-f856-45f9-ad34-c5e6afad8850'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.271173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcf4d7dd1c354906beb30b1d6d9e716d', 'request_id': 'req-977c113e-f856-45f9-ad34-c5e6afad8850'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.297303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:41.314446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] The token cache is being invalidate because project bcf4d7dd1c354906beb30b1d6d9e716d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:41.314744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.316028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.322365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 934/1849] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:41 2022] DELETE /identity/v3/projects/bcf4d7dd1c354906beb30b1d6d9e716d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:41.328881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:41.332983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.380760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 916/1850] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:41.384514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:41.385866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.425495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c113e-f856-45f9-ad34-c5e6afad8850 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:41.434212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:41.434902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:41.435475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:41.435475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] PATH_INFO: `/v3/users/e8bd092e54984669b587347b0de5a6c5` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:41.436641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8bd092e54984669b587347b0de5a6c5)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:41.439992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:41.457809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba20fe-1a70-43cf-a1a6-b7b0320a6127 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:41.463197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:41.463971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:41.464362 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:41.464768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:41.467016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.502862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8bd092e54984669b587347b0de5a6c5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.514715 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8bd092e54984669b587347b0de5a6c5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.515478 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8bd092e54984669b587347b0de5a6c5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.544729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8bd092e54984669b587347b0de5a6c5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.554982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8bd092e54984669b587347b0de5a6c5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:41.561479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 935/1851] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:41 2022] DELETE /identity/v3/users/e8bd092e54984669b587347b0de5a6c5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:41.562289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:41.563678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.587215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:41.588936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:41.606081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 917/1852] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:41.659572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3160a-72e9-494b-be03-5f0418bc06e1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:41.666579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1705c93-9671-4adf-9ae1-0fa3807007ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:41.667387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1705c93-9671-4adf-9ae1-0fa3807007ad service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:41.669876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1705c93-9671-4adf-9ae1-0fa3807007ad service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:41.670165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1705c93-9671-4adf-9ae1-0fa3807007ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:41.672546 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c1705c93-9671-4adf-9ae1-0fa3807007ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:41.684813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:41.792014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1705c93-9671-4adf-9ae1-0fa3807007ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:41.795832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1705c93-9671-4adf-9ae1-0fa3807007ad service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:41.821388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 936/1853] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:42.272746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:42.295489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ed275-a4c1-4bc6-b323-b5fccc46b299 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:42.331221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:42.333436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.337835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.338545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] PATH_INFO: `/v3/projects/64c62a980ade402e843a31bedd35c693` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.344196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c62a980ade402e843a31bedd35c693)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:42.353007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:42.451416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c62a980ade402e843a31bedd35c693', 'request_id': 'req-6f0b8aa9-a7ab-4350-94ca-151697608e85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:42.459360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c62a980ade402e843a31bedd35c693', 'request_id': 'req-6f0b8aa9-a7ab-4350-94ca-151697608e85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:42.472020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c62a980ade402e843a31bedd35c693', 'request_id': 'req-6f0b8aa9-a7ab-4350-94ca-151697608e85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:42.486764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] The token cache is being invalidate because project 64c62a980ade402e843a31bedd35c693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:42.487316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:42.488288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6f0b8aa9-a7ab-4350-94ca-151697608e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:42.492869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 918/1854] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:42 2022] DELETE /identity/v3/projects/64c62a980ade402e843a31bedd35c693 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:42.498066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d920161-80bb-4826-8c18-7908b2e1b2ad None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.498905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d920161-80bb-4826-8c18-7908b2e1b2ad None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.499710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d920161-80bb-4826-8c18-7908b2e1b2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.529742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9d920161-80bb-4826-8c18-7908b2e1b2ad None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:42.531311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9d920161-80bb-4826-8c18-7908b2e1b2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:42.631159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 937/1855] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:42.638582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86df7-c073-4740-8f1c-d750b7989c13 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.638990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86df7-c073-4740-8f1c-d750b7989c13 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.639400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86df7-c073-4740-8f1c-d750b7989c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.649058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:42.662642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bce86df7-c073-4740-8f1c-d750b7989c13 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:42.664837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bce86df7-c073-4740-8f1c-d750b7989c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:42.673437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 919/1856] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:42.678220 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d920161-80bb-4826-8c18-7908b2e1b2ad None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:42.689584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d920161-80bb-4826-8c18-7908b2e1b2ad None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:42.696396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:42.697026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.697365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.697723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.698389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:42.698911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:42.700568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:42.715637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 938/1857] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:42.719812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86df7-c073-4740-8f1c-d750b7989c13 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:42.730668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86df7-c073-4740-8f1c-d750b7989c13 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:42.736346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160ad5f-83a4-437a-907e-8d54461b503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:42.737282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160ad5f-83a4-437a-907e-8d54461b503c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.737401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160ad5f-83a4-437a-907e-8d54461b503c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.737700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160ad5f-83a4-437a-907e-8d54461b503c admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.738465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160ad5f-83a4-437a-907e-8d54461b503c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:42.740936 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160ad5f-83a4-437a-907e-8d54461b503c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:42.778496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 920/1858] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:42 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:42.779568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:42.793767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac51ea4-9f9e-453f-9c7d-a9f431d755bd admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:42.853235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.853610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.853965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.879336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:42.880033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.880263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.880534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.881493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:42.884253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:42.906723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] MFA Rules not processed for user `67f6989c3c4d4fe3a2cc3be83a5f5f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:42.908315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:42.915161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 939/1859] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:42 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:42.917439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:42.939105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2502d-8250-4755-853a-4238ba80c1a8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:42.946212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:42.947223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:42.947694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:42.948214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:42.949114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:42.951652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:42.991685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 940/1860] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:42.995299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.019208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a683db-5690-413f-acb2-c0e895e9ed95 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.023762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 921/1861] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:43.025179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.028264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:43.029699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.030743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.031196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.031677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.032632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.035865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.042700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:43.048751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c5e80-f10d-4f82-978b-479b66498d33 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.061679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 941/1862] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:43.062933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.064547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.073800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.075677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.076111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.077736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.080667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d182946-23af-41d3-95ad-27a6d34afc42 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.083887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.099530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92db3b-e543-443e-9a0d-f0a5d8326fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.100408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92db3b-e543-443e-9a0d-f0a5d8326fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.100701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92db3b-e543-443e-9a0d-f0a5d8326fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.100987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92db3b-e543-443e-9a0d-f0a5d8326fc2 admin admin] PATH_INFO: `/v3/projects/bf065d0e0ce44345ad50a9f82d246a3a/users/55c829e63c86452ea544139587565526/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.166264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92db3b-e543-443e-9a0d-f0a5d8326fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf065d0e0ce44345ad50a9f82d246a3a, user_id=55c829e63c86452ea544139587565526, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.175710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92db3b-e543-443e-9a0d-f0a5d8326fc2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.187128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 922/1863] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:43 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:43.203343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.211337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 942/1864] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:43 2022] PUT /identity/v3/projects/bf065d0e0ce44345ad50a9f82d246a3a/users/55c829e63c86452ea544139587565526/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:43.222539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.223000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.223460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.333463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] MFA Rules not processed for user `55c829e63c86452ea544139587565526`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:43.335935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:43.392369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692e57-adbb-4770-bc2d-f4b90b224aa3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.517968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:43.527197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 943/1865] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:43.538063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.538779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.539251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.539540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.540208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.542037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.543482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:43.543843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.584002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 923/1866] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:43 2022] POST /identity/v3/users => generated 292 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:43.592899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.606406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255d5d-bbd3-4d51-bb7f-daa506709556 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.611375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.613373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.613658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.614043 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.615279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.617621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.631203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81a4d-b09d-47c3-aa10-a57f1705f555 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.631203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.631203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.631203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.631681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.641597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 924/1867] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:43.642594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.645065 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.646516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.655612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03269f-6649-43ac-a2e1-c316ec064aa4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.658382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 944/1868] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:08:43.661965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.663030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.663797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.664656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] PATH_INFO: `/v3/projects/25eaf46184414a03951c8287c890619f/users/2f1c1e246b574f1e854cc6d2849b46bc/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.690006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25eaf46184414a03951c8287c890619f, user_id=2f1c1e246b574f1e854cc6d2849b46bc, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.693287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.736028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 925/1869] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:43 2022] PUT /identity/v3/projects/25eaf46184414a03951c8287c890619f/users/2f1c1e246b574f1e854cc6d2849b46bc/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:43.740038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.759554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004ce58-d55e-4e3d-9568-38a51faa9924 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.868045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777628c-bd22-4236-beb8-b86c4a718a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.868986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777628c-bd22-4236-beb8-b86c4a718a81 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.869594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777628c-bd22-4236-beb8-b86c4a718a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.870054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777628c-bd22-4236-beb8-b86c4a718a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.870758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777628c-bd22-4236-beb8-b86c4a718a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.872542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777628c-bd22-4236-beb8-b86c4a718a81 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.887207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 945/1870] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:43.891896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:43.903625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f28c4-522d-418d-a134-5a1d03a1b390 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:43.911372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:43.912144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:43.912607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:43.913027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] PATH_INFO: `/v3/projects/25eaf46184414a03951c8287c890619f/users/2f1c1e246b574f1e854cc6d2849b46bc/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:43.917710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25eaf46184414a03951c8287c890619f, user_id=2f1c1e246b574f1e854cc6d2849b46bc, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:43.920713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:43.933926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 926/1871] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:43 2022] PUT /identity/v3/projects/25eaf46184414a03951c8287c890619f/users/2f1c1e246b574f1e854cc6d2849b46bc/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:44.740226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:44.740709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:44.741136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:44.752503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:44.784632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] MFA Rules not processed for user `2f1c1e246b574f1e854cc6d2849b46bc`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:44.786087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:44.874946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 946/1872] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:44.891470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:44.893209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:45.007056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588aa8-6aaa-4210-988d-0319e012e25e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:45.017294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a9f4-689a-4cfa-8329-5b4ce4312de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:45.018146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a9f4-689a-4cfa-8329-5b4ce4312de7 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:45.018734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a9f4-689a-4cfa-8329-5b4ce4312de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:45.019263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a9f4-689a-4cfa-8329-5b4ce4312de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:45.082507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:45.084020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:45.153540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a9f4-689a-4cfa-8329-5b4ce4312de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:45.155423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a9f4-689a-4cfa-8329-5b4ce4312de7 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:45.169548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 927/1873] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:45.292266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faae843-e6b8-42c8-b6eb-a4dd8e2205f4 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:45.303115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:45.303789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:45.304069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:45.304431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] PATH_INFO: `/v3/users/590251bf628a48c88dccb2827af1a1d6` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:45.308727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=590251bf628a48c88dccb2827af1a1d6)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:45.310786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:45.404564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590251bf628a48c88dccb2827af1a1d6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:45.421996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590251bf628a48c88dccb2827af1a1d6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:45.422791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590251bf628a48c88dccb2827af1a1d6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:45.448819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590251bf628a48c88dccb2827af1a1d6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:45.465951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590251bf628a48c88dccb2827af1a1d6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:45.477291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 947/1874] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:45 2022] DELETE /identity/v3/users/590251bf628a48c88dccb2827af1a1d6 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:45.481891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:45.482401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:45.482743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:45.519736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:45.524492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:45.610916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:45.638597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 928/1875] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:45.649938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:45.652119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:45.669230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:45.758041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243267cb-533d-4e89-8043-4e1423e6dc2f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:45.762453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:45.763181 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:45.763526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:45.763865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:45.774893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:45.776493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:45.789283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 948/1876] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:46.256588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:46.259325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:46.295851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a95d5-5832-432c-8ca6-7c1bee968728 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:46.306040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:46.306977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:46.307451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:46.307751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] PATH_INFO: `/v3/projects/74fd78a2177d4e83a644a528669825f7` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:46.308939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74fd78a2177d4e83a644a528669825f7)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:46.310963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:46.368688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74fd78a2177d4e83a644a528669825f7', 'request_id': 'req-6be8c14c-7490-45f5-9128-10c86646bf99'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.378137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74fd78a2177d4e83a644a528669825f7', 'request_id': 'req-6be8c14c-7490-45f5-9128-10c86646bf99'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.389744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74fd78a2177d4e83a644a528669825f7', 'request_id': 'req-6be8c14c-7490-45f5-9128-10c86646bf99'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.413108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] The token cache is being invalidate because project 74fd78a2177d4e83a644a528669825f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:46.413400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.419581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.421357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 929/1877] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:46 2022] DELETE /identity/v3/projects/74fd78a2177d4e83a644a528669825f7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:46.425765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:46.439474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c3de5-0342-40f9-8b05-2dfea4c2ea27 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:46.542854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:46.544046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:46.544427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:46.544712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] PATH_INFO: `/v3/users/dc4a72a69aa24c1cb0ac56b456ec77f2` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:46.546046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc4a72a69aa24c1cb0ac56b456ec77f2)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:46.548366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:46.617794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc4a72a69aa24c1cb0ac56b456ec77f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.629362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc4a72a69aa24c1cb0ac56b456ec77f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.630038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc4a72a69aa24c1cb0ac56b456ec77f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.643917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc4a72a69aa24c1cb0ac56b456ec77f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.652563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9c60eae6-882f-46f9-b806-a29663440df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc4a72a69aa24c1cb0ac56b456ec77f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:46.660738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 949/1878] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:46 2022] DELETE /identity/v3/users/dc4a72a69aa24c1cb0ac56b456ec77f2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:47.009346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.010921 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:47.040304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c14c-7490-45f5-9128-10c86646bf99 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.122636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.123095 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.123464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.123816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] PATH_INFO: `/v3/projects/48617c39cab8430cac35ca731c4988a2` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.125328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48617c39cab8430cac35ca731c4988a2)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.127377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.193055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48617c39cab8430cac35ca731c4988a2', 'request_id': 'req-15448e4a-733a-4e70-a4c4-4e5dad731253'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:47.203715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48617c39cab8430cac35ca731c4988a2', 'request_id': 'req-15448e4a-733a-4e70-a4c4-4e5dad731253'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:47.215820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48617c39cab8430cac35ca731c4988a2', 'request_id': 'req-15448e4a-733a-4e70-a4c4-4e5dad731253'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:47.223158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:47.227883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] The token cache is being invalidate because project 48617c39cab8430cac35ca731c4988a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:47.228633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:47.229897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-15448e4a-733a-4e70-a4c4-4e5dad731253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:47.232120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 930/1879] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:47 2022] DELETE /identity/v3/projects/48617c39cab8430cac35ca731c4988a2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:47.237025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e16c-b92e-4716-96ce-a29f1c27a3a0 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.237585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e16c-b92e-4716-96ce-a29f1c27a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.237712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e16c-b92e-4716-96ce-a29f1c27a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.258962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-57e1e16c-b92e-4716-96ce-a29f1c27a3a0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:47.260230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-57e1e16c-b92e-4716-96ce-a29f1c27a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:47.288774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:47.327629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 950/1880] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:47.336247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e575-8210-4a9a-bcc5-2fb91801e2c7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.336778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e575-8210-4a9a-bcc5-2fb91801e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.337077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e575-8210-4a9a-bcc5-2fb91801e2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.358551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-90e3e575-8210-4a9a-bcc5-2fb91801e2c7 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:47.359851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-90e3e575-8210-4a9a-bcc5-2fb91801e2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:47.373799 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 931/1881] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:47.378259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1e16c-b92e-4716-96ce-a29f1c27a3a0 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.388877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1e16c-b92e-4716-96ce-a29f1c27a3a0 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.395658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.396473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.396818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.397096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.397763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:47.398069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.401427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.417391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 951/1882] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:47.421812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e575-8210-4a9a-bcc5-2fb91801e2c7 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.433157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e575-8210-4a9a-bcc5-2fb91801e2c7 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.437992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.438697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.438984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.439295 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.439944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.441887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.472785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 932/1883] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:47 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:47.477187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.487662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c27cc-1dee-41e1-aaae-f4ac03f79bde admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.553440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.553729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.553954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.554775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.555871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.557839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.588352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 952/1884] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:47 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:47.592554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.603926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e137097-b5dd-42f3-ad93-98a755030326 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.608509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.609272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.609660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.609941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.610599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.612251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.624839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 933/1885] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:47.628775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.640087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9e5d7-48e6-45e1-bf1f-25d2ba8c7888 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.645593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd20719-a261-48f0-aba6-f32d2bb27782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.646206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20719-a261-48f0-aba6-f32d2bb27782 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.646492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20719-a261-48f0-aba6-f32d2bb27782 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.646768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20719-a261-48f0-aba6-f32d2bb27782 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.647650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd20719-a261-48f0-aba6-f32d2bb27782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.649343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd20719-a261-48f0-aba6-f32d2bb27782 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.664845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 953/1886] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:47.665687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.676724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919869d-b7b5-4f15-879f-c171120fc9c8 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.683394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.684076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.684355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.684633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] PATH_INFO: `/v3/projects/de940d51595e44f0baa640245728362a/users/428685d79cfe44afa3eb7a8da9844f47/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.702136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=de940d51595e44f0baa640245728362a, user_id=428685d79cfe44afa3eb7a8da9844f47, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.704998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.724965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 934/1887] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:47 2022] PUT /identity/v3/projects/de940d51595e44f0baa640245728362a/users/428685d79cfe44afa3eb7a8da9844f47/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:47.737000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e7aa-0ea4-43da-8486-7a16b50e015e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.737291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e7aa-0ea4-43da-8486-7a16b50e015e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.737565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e7aa-0ea4-43da-8486-7a16b50e015e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.775331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b311e7aa-0ea4-43da-8486-7a16b50e015e None None] MFA Rules not processed for user `428685d79cfe44afa3eb7a8da9844f47`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:47.776784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b311e7aa-0ea4-43da-8486-7a16b50e015e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:47.861986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 954/1888] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:47.866526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.932339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba41ac1-2283-4018-ac14-ed31133621dd admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:47.937202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:47.937822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:47.938108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:47.938382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:47.939041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:47.941023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:47.971132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 935/1889] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:47 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:47.975432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311e7aa-0ea4-43da-8486-7a16b50e015e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:47.986833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311e7aa-0ea4-43da-8486-7a16b50e015e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.013281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:48.050254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.050877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.051196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.051476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.052115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.054073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.080582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 955/1890] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:47 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:48.081785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:48.093958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1c2f5-ef49-4eb7-9854-ce48ce832bb4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.098622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.099271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.099553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.099828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.100510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.102292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.113395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 936/1891] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:48.118450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:48.129064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1633e-7afa-4de0-bd89-81b88e623c35 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.135175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.135841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.136119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.136394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] PATH_INFO: `/v3/projects/0b9fe03fe6834c5f8a87e3ad1ea83471/users/fd3e26307f4a43a0948319670f3e2cf7/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.155613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9fe03fe6834c5f8a87e3ad1ea83471, user_id=fd3e26307f4a43a0948319670f3e2cf7, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.158345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.173140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 956/1892] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:48 2022] PUT /identity/v3/projects/0b9fe03fe6834c5f8a87e3ad1ea83471/users/fd3e26307f4a43a0948319670f3e2cf7/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:48.177266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:48.188766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f161-ca5d-440e-8b77-225ac5879e8d admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.256367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccaf99-049b-40b4-9e02-8e5531578142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.257180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccaf99-049b-40b4-9e02-8e5531578142 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.257777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccaf99-049b-40b4-9e02-8e5531578142 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.258485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccaf99-049b-40b4-9e02-8e5531578142 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.259974 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccaf99-049b-40b4-9e02-8e5531578142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.261897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccaf99-049b-40b4-9e02-8e5531578142 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.272862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 937/1893] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:48.277767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:48.289173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092c28-c86d-42b6-8a68-b2e1be2c645d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.294092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.294772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.295049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.295361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] PATH_INFO: `/v3/projects/0b9fe03fe6834c5f8a87e3ad1ea83471/users/fd3e26307f4a43a0948319670f3e2cf7/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.297303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9fe03fe6834c5f8a87e3ad1ea83471, user_id=fd3e26307f4a43a0948319670f3e2cf7, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.300113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.316778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 957/1894] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:48 2022] PUT /identity/v3/projects/0b9fe03fe6834c5f8a87e3ad1ea83471/users/fd3e26307f4a43a0948319670f3e2cf7/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:48.327855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777d14-e5ad-4696-8149-73bad4969a56 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.328138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777d14-e5ad-4696-8149-73bad4969a56 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.328419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777d14-e5ad-4696-8149-73bad4969a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.344002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:48.360535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f4777d14-e5ad-4696-8149-73bad4969a56 None None] MFA Rules not processed for user `fd3e26307f4a43a0948319670f3e2cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:48.361874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f4777d14-e5ad-4696-8149-73bad4969a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:48.443532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 938/1895] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:48.455784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:48.457433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:48.515785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9db80-2e7f-4f21-9df1-afe0068cb9c0 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.519288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c8477-3a5b-4bf9-98fc-730256b5d8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.519882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c8477-3a5b-4bf9-98fc-730256b5d8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.520157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c8477-3a5b-4bf9-98fc-730256b5d8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.520424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c8477-3a5b-4bf9-98fc-730256b5d8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.530971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c8477-3a5b-4bf9-98fc-730256b5d8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.532508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c8477-3a5b-4bf9-98fc-730256b5d8b1 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.542891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 958/1896] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:48.571350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4777d14-e5ad-4696-8149-73bad4969a56 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:48.640329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4777d14-e5ad-4696-8149-73bad4969a56 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.646005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.646644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.646927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.647239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] PATH_INFO: `/v3/users/428685d79cfe44afa3eb7a8da9844f47` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.649269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=428685d79cfe44afa3eb7a8da9844f47)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.651863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.702597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '428685d79cfe44afa3eb7a8da9844f47'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:48.715549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '428685d79cfe44afa3eb7a8da9844f47'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:48.716276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '428685d79cfe44afa3eb7a8da9844f47'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:48.716908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '428685d79cfe44afa3eb7a8da9844f47'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:48.723774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '428685d79cfe44afa3eb7a8da9844f47'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:48.736618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 939/1897] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:48 2022] DELETE /identity/v3/users/428685d79cfe44afa3eb7a8da9844f47 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:48.741532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.741787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.742135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.761934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:48.763176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:48.826574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 959/1898] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:48.838289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:48.839761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:48.857654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:48.927978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c2871-41de-4a8a-affa-8f3079c64ca0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:48.933607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:48.934510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:48.934943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:48.935280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:48.946312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:48.947819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:48.961129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 940/1899] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:49.240624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:49.243287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:49.255549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540331e-d75d-4b56-bfb2-26d68aba7054 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:49.261028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:49.261670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:49.261948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:49.262220 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] PATH_INFO: `/v3/projects/de940d51595e44f0baa640245728362a` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:49.263359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=de940d51595e44f0baa640245728362a)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:49.265311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:49.284884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:49.318300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de940d51595e44f0baa640245728362a', 'request_id': 'req-a1173e07-8dfc-4029-a1df-6481aa6386ab'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.325786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de940d51595e44f0baa640245728362a', 'request_id': 'req-a1173e07-8dfc-4029-a1df-6481aa6386ab'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.332076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de940d51595e44f0baa640245728362a', 'request_id': 'req-a1173e07-8dfc-4029-a1df-6481aa6386ab'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.344635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] The token cache is being invalidate because project de940d51595e44f0baa640245728362a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:49.344926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.345552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.347053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 960/1900] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:49 2022] DELETE /identity/v3/projects/de940d51595e44f0baa640245728362a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:49.351444 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:49.361161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7767347-534f-408a-b52c-8c88fc8297d3 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:49.423830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:49.424518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:49.424773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:49.425035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] PATH_INFO: `/v3/users/fd3e26307f4a43a0948319670f3e2cf7` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:49.426156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3e26307f4a43a0948319670f3e2cf7)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:49.428135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:49.481010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3e26307f4a43a0948319670f3e2cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.494914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3e26307f4a43a0948319670f3e2cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.495578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3e26307f4a43a0948319670f3e2cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.496185 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3e26307f4a43a0948319670f3e2cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.503039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3e26307f4a43a0948319670f3e2cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:49.518669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 941/1901] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:49 2022] DELETE /identity/v3/users/fd3e26307f4a43a0948319670f3e2cf7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:49.871320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:49.875284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:49.889128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1173e07-8dfc-4029-a1df-6481aa6386ab admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:49.963529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:49.964216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:49.964448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:49.964766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] PATH_INFO: `/v3/projects/0b9fe03fe6834c5f8a87e3ad1ea83471` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:49.970536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9fe03fe6834c5f8a87e3ad1ea83471)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:49.970536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:49.988312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:49.989750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:50.031440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9fe03fe6834c5f8a87e3ad1ea83471', 'request_id': 'req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:50.042277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9fe03fe6834c5f8a87e3ad1ea83471', 'request_id': 'req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:50.051030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:50.062583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9fe03fe6834c5f8a87e3ad1ea83471', 'request_id': 'req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:50.090982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] The token cache is being invalidate because project 0b9fe03fe6834c5f8a87e3ad1ea83471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:50.092871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:50.094319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-19d5bb3d-f11b-485e-b8ad-4991acad2aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:50.099414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586fcb3-a04c-436a-985a-8ea4cdfd9048 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.102016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 961/1902] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:49 2022] DELETE /identity/v3/projects/0b9fe03fe6834c5f8a87e3ad1ea83471 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:50.105223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58af42c-e97b-4acd-8aa6-a02095f2a46c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.105551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58af42c-e97b-4acd-8aa6-a02095f2a46c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.106192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58af42c-e97b-4acd-8aa6-a02095f2a46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.136829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d58af42c-e97b-4acd-8aa6-a02095f2a46c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:50.138714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d58af42c-e97b-4acd-8aa6-a02095f2a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:50.191250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:50.207991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.209582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.209966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.210437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.217636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:50.260975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 962/1903] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:50.271854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00554d64-ddbe-4a7d-8469-977030a54529 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.272621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00554d64-ddbe-4a7d-8469-977030a54529 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.273948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00554d64-ddbe-4a7d-8469-977030a54529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.303792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-00554d64-ddbe-4a7d-8469-977030a54529 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:50.305378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-00554d64-ddbe-4a7d-8469-977030a54529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:50.314524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 963/1904] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:50.318683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00554d64-ddbe-4a7d-8469-977030a54529 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.329781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00554d64-ddbe-4a7d-8469-977030a54529 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.335008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.336036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.336411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.336725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.337381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:50.337686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.339431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.353163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 964/1905] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:50.357614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.359305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.360920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.371364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80e41e-2ddb-423f-8112-3bfc9e911a08 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.377908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 942/1906] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:08:50.387096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.388027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.388481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.389003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.389904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.392485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.443503 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 965/1907] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:50 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:50.444809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.455743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36612d3-9cba-4bfe-9fa7-1aacb2f61cd2 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.520792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.521500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.521788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.522065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.522708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.525452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.555981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 943/1908] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:50 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:50.560343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.572187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8d4d8-321c-4edf-b0d1-4885839afedc admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.577146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.577757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.578040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.578450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.579131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.581096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.595228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 966/1909] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:50.599877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.614724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ce021-0b55-4e33-b424-fcdfbe688580 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.623451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f75ef-41a2-4cfb-8951-a69a6f476a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.624326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f75ef-41a2-4cfb-8951-a69a6f476a22 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.625297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f75ef-41a2-4cfb-8951-a69a6f476a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.625297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f75ef-41a2-4cfb-8951-a69a6f476a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.625803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f75ef-41a2-4cfb-8951-a69a6f476a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.627563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f75ef-41a2-4cfb-8951-a69a6f476a22 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.649888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 944/1910] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:50.655599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.666919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538650e-6ca3-4abc-a652-dacd1659b2c3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.673233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.674371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.675282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.675633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] PATH_INFO: `/v3/projects/92fce9b61cca4a0f8c69ac55f1703ecb/users/4aaa84a86fe34c5abdb64433ec3a6334/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.703900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92fce9b61cca4a0f8c69ac55f1703ecb, user_id=4aaa84a86fe34c5abdb64433ec3a6334, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.706673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.722907 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 967/1911] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:50 2022] PUT /identity/v3/projects/92fce9b61cca4a0f8c69ac55f1703ecb/users/4aaa84a86fe34c5abdb64433ec3a6334/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:50.737083 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0b2c6-5385-40a5-99f3-33fc3789a14e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.737377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0b2c6-5385-40a5-99f3-33fc3789a14e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.737665 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0b2c6-5385-40a5-99f3-33fc3789a14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.771057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8ef0b2c6-5385-40a5-99f3-33fc3789a14e None None] MFA Rules not processed for user `4aaa84a86fe34c5abdb64433ec3a6334`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:50.772256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8ef0b2c6-5385-40a5-99f3-33fc3789a14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:50.777896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:50.856953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 945/1912] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:50.861243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.928367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fe449-d5e3-43a0-8d97-fe0f672f13e6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:50.934502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:50.935331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:50.935735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:50.935988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:50.936701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:50.938760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:50.971529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 968/1913] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:50 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:50.974910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0b2c6-5385-40a5-99f3-33fc3789a14e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:50.991804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0b2c6-5385-40a5-99f3-33fc3789a14e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:51.059318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:51.059994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:51.060272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:51.060544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:51.061179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:51.063180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:51.088346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 946/1914] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:50 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:51.092525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:51.103868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b42a4-caa9-4a5e-bc1e-8f6eeadd20a9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:51.108923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:51.109668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:51.110008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:51.110382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:51.111031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:51.113018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:51.125590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 969/1915] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:51.130306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:51.140995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7f398-180b-425a-a03c-80df1dde9b17 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:51.145707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:51.146376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:51.146656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:51.146930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] PATH_INFO: `/v3/projects/3b01e57bc5a9438fae54f99140d8eabf/users/5fb3cb080aea4902b9999a9270377bc3/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:51.174318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b01e57bc5a9438fae54f99140d8eabf, user_id=5fb3cb080aea4902b9999a9270377bc3, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:51.174318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:51.185523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 947/1916] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:51 2022] PUT /identity/v3/projects/3b01e57bc5a9438fae54f99140d8eabf/users/5fb3cb080aea4902b9999a9270377bc3/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:51.189460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:51.200633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635be822-fec0-4aaf-9032-be5e30254955 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:51.231372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:51.264238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b576c-7b5f-4c05-937b-85daebad0d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:51.264854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b576c-7b5f-4c05-937b-85daebad0d64 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:51.265136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b576c-7b5f-4c05-937b-85daebad0d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:51.265440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b576c-7b5f-4c05-937b-85daebad0d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:51.266084 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b576c-7b5f-4c05-937b-85daebad0d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:51.268045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b576c-7b5f-4c05-937b-85daebad0d64 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:51.281162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 970/1917] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:51.285387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:51.296069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9834db4-ddd0-46f5-a28e-5fef1c1610b1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:51.300847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:51.301573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:51.301861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:51.302138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] PATH_INFO: `/v3/projects/3b01e57bc5a9438fae54f99140d8eabf/users/5fb3cb080aea4902b9999a9270377bc3/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:51.304128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b01e57bc5a9438fae54f99140d8eabf, user_id=5fb3cb080aea4902b9999a9270377bc3, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:51.307469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:51.318050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 948/1918] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:51 2022] PUT /identity/v3/projects/3b01e57bc5a9438fae54f99140d8eabf/users/5fb3cb080aea4902b9999a9270377bc3/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:51.328819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ba5b3-3b3b-4c0a-86b0-296db2200a77 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:51.329313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ba5b3-3b3b-4c0a-86b0-296db2200a77 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:51.329593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ba5b3-3b3b-4c0a-86b0-296db2200a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:51.358361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-bf4ba5b3-3b3b-4c0a-86b0-296db2200a77 None None] MFA Rules not processed for user `5fb3cb080aea4902b9999a9270377bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:51.359599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bf4ba5b3-3b3b-4c0a-86b0-296db2200a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:51.433769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 971/1919] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:51.444591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:51.447221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:51.543109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaef52-9cbd-42a3-adc8-18b143342a6b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:51.547698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4a16-daa3-43ec-82bc-94bfcf79b4de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:51.549020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4a16-daa3-43ec-82bc-94bfcf79b4de service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:51.549391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4a16-daa3-43ec-82bc-94bfcf79b4de service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:51.549742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4a16-daa3-43ec-82bc-94bfcf79b4de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:51.561786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4a16-daa3-43ec-82bc-94bfcf79b4de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:51.563331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4a16-daa3-43ec-82bc-94bfcf79b4de service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:51.578378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 949/1920] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:52.661280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ba5b3-3b3b-4c0a-86b0-296db2200a77 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:52.750379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ba5b3-3b3b-4c0a-86b0-296db2200a77 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:52.755469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:52.756111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:52.756398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:52.756670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] PATH_INFO: `/v3/users/4aaa84a86fe34c5abdb64433ec3a6334` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:52.758176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aaa84a86fe34c5abdb64433ec3a6334)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:52.760202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:52.823772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aaa84a86fe34c5abdb64433ec3a6334'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:52.837609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aaa84a86fe34c5abdb64433ec3a6334'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:52.838293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aaa84a86fe34c5abdb64433ec3a6334'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:52.841175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:52.856638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aaa84a86fe34c5abdb64433ec3a6334'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:52.864764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aaa84a86fe34c5abdb64433ec3a6334'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:52.871799 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 972/1921] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:52 2022] DELETE /identity/v3/users/4aaa84a86fe34c5abdb64433ec3a6334 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:52.877277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:52.877752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:52.877853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:52.885509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:52.902041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:52.903444 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:52.965266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 950/1922] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:52.975153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:52.977172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:53.039145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ebd01-9817-4165-9d9a-feebfe88f2f8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:53.042742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4193a-9a53-4007-adea-acd426fb27c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:53.043383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4193a-9a53-4007-adea-acd426fb27c3 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:53.043662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4193a-9a53-4007-adea-acd426fb27c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:53.043933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4193a-9a53-4007-adea-acd426fb27c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:53.053270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4193a-9a53-4007-adea-acd426fb27c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:53.055735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4193a-9a53-4007-adea-acd426fb27c3 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:53.067061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 973/1923] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:53.146097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:53.147788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:53.164233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12593ff4-e5af-4430-93f5-003fc4f5a272 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:53.169261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:53.169939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:53.170220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:53.170661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] PATH_INFO: `/v3/users/55c829e63c86452ea544139587565526` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:53.171919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c829e63c86452ea544139587565526)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:53.173842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:53.235595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c829e63c86452ea544139587565526'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.260292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c829e63c86452ea544139587565526'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.261307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c829e63c86452ea544139587565526'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.261937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c829e63c86452ea544139587565526'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.274657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c829e63c86452ea544139587565526'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.296108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 951/1924] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:53 2022] DELETE /identity/v3/users/55c829e63c86452ea544139587565526 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:53.301325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:53.301613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:53.301900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:53.332030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:53.334190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:53.465616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 974/1925] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:53.475226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:53.477551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:53.483035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:53.484712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:53.504369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaa8cf-772e-4792-b145-08bb7c8f3f4a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:53.510758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:53.511594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:53.511888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:53.512166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] PATH_INFO: `/v3/projects/92fce9b61cca4a0f8c69ac55f1703ecb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:53.513580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92fce9b61cca4a0f8c69ac55f1703ecb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:53.515771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:53.595237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb185a6-7722-4555-84fa-3e74d01320a5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:53.606732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92fce9b61cca4a0f8c69ac55f1703ecb', 'request_id': 'req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.617848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92fce9b61cca4a0f8c69ac55f1703ecb', 'request_id': 'req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.630930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92fce9b61cca4a0f8c69ac55f1703ecb', 'request_id': 'req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.644582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:53.664292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] The token cache is being invalidate because project 92fce9b61cca4a0f8c69ac55f1703ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:53.665669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.667726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.670689 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:53.674172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 975/1926] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:53 2022] DELETE /identity/v3/projects/92fce9b61cca4a0f8c69ac55f1703ecb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:53.675854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:53.690571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:53.694180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:53.697127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:53.697508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:53.697748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:53.701497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:53.809833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e56d3a-7d8e-4e8b-9038-277bd587a8c9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:53.816064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:53.816498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:53.817367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:53.817725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:53.818072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] PATH_INFO: `/v3/users/5fb3cb080aea4902b9999a9270377bc3` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:53.818693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:53.819422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb3cb080aea4902b9999a9270377bc3)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:53.821765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:53.839229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 952/1927] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:53.912576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb3cb080aea4902b9999a9270377bc3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.921661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb3cb080aea4902b9999a9270377bc3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.923033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb3cb080aea4902b9999a9270377bc3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.940676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb3cb080aea4902b9999a9270377bc3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.949211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb3cb080aea4902b9999a9270377bc3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:53.956185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 976/1928] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:53 2022] DELETE /identity/v3/users/5fb3cb080aea4902b9999a9270377bc3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:54.325434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:54.331865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:54.355273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c80e3-0741-40de-98d9-d5beb7b674cb service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:54.441327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:54.442895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:54.474386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:54.475990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.476905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.477308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] PATH_INFO: `/v3/projects/bf065d0e0ce44345ad50a9f82d246a3a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.479498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf065d0e0ce44345ad50a9f82d246a3a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:54.481852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:54.525076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef4f6f-c2bc-4988-bec3-8a43ffdb38be admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:54.530217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:54.531149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.531484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.531711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] PATH_INFO: `/v3/projects/3b01e57bc5a9438fae54f99140d8eabf` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.532837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b01e57bc5a9438fae54f99140d8eabf)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:54.534819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:54.565038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf065d0e0ce44345ad50a9f82d246a3a', 'request_id': 'req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.574582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf065d0e0ce44345ad50a9f82d246a3a', 'request_id': 'req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.580363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:54.586952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf065d0e0ce44345ad50a9f82d246a3a', 'request_id': 'req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.604303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] The token cache is being invalidate because project bf065d0e0ce44345ad50a9f82d246a3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:54.604611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.605889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a48cc124-50a3-4598-bf2d-ac926b6f0ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.611313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 953/1929] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:54 2022] DELETE /identity/v3/projects/bf065d0e0ce44345ad50a9f82d246a3a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:54.612749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65e985-44ae-4524-bc44-a84564b8fc49 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.613127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65e985-44ae-4524-bc44-a84564b8fc49 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.613478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65e985-44ae-4524-bc44-a84564b8fc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.616133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b01e57bc5a9438fae54f99140d8eabf', 'request_id': 'req-1876657c-558b-42cf-bcf5-b82b862c98e2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.623694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b01e57bc5a9438fae54f99140d8eabf', 'request_id': 'req-1876657c-558b-42cf-bcf5-b82b862c98e2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.629936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b01e57bc5a9438fae54f99140d8eabf', 'request_id': 'req-1876657c-558b-42cf-bcf5-b82b862c98e2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.637794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1d65e985-44ae-4524-bc44-a84564b8fc49 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:54.639033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1d65e985-44ae-4524-bc44-a84564b8fc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:54.645919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] The token cache is being invalidate because project 3b01e57bc5a9438fae54f99140d8eabf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:54.646119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:54.646576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.647369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1876657c-558b-42cf-bcf5-b82b862c98e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:54.650041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 977/1930] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:54 2022] DELETE /identity/v3/projects/3b01e57bc5a9438fae54f99140d8eabf => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:54.656995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a005d79-25d7-4246-b794-5c979d6dad74 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.657408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a005d79-25d7-4246-b794-5c979d6dad74 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.657779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a005d79-25d7-4246-b794-5c979d6dad74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.682645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1a005d79-25d7-4246-b794-5c979d6dad74 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:54.684863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1a005d79-25d7-4246-b794-5c979d6dad74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:54.718492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 954/1931] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:54.727569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee344-c453-45df-b656-0de8d82bd53e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.727569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee344-c453-45df-b656-0de8d82bd53e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.727818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee344-c453-45df-b656-0de8d82bd53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.769056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 978/1932] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:54.770601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77fd8c-9598-418c-9099-f9b72ec32374 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.771342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77fd8c-9598-418c-9099-f9b72ec32374 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.771957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77fd8c-9598-418c-9099-f9b72ec32374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.795707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2b77fd8c-9598-418c-9099-f9b72ec32374 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:54.797315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2b77fd8c-9598-418c-9099-f9b72ec32374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:54.805249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-daeee344-c453-45df-b656-0de8d82bd53e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:54.806650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-daeee344-c453-45df-b656-0de8d82bd53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:54.814858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 979/1933] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:54.815930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77fd8c-9598-418c-9099-f9b72ec32374 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:54.827254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77fd8c-9598-418c-9099-f9b72ec32374 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:54.833444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:54.834372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.834973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.835926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.837223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:54.837833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 955/1934] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:54.838292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:54.840743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:54.843727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeee344-c453-45df-b656-0de8d82bd53e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:54.858783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeee344-c453-45df-b656-0de8d82bd53e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:54.867703 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 980/1935] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:54.868885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:54.874329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:54.874960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.875268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.875545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.876207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:54.876662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:54.878283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:54.894641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53548d4f-3759-4888-b3f2-d9a9af8fd05f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:54.906037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:54.906850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.907505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.909209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.910346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:54.916867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:54.931006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 956/1936] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:54.943373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:54.962860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d325e-aa49-4a7c-bf01-231f1a0b715a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:54.979580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:54.980209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:54.980493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:54.980770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:54.981422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:54.986045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 981/1937] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:54 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:54.987771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:54.988792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.041832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 957/1938] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:54 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:55.042938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.141461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787faf57-c848-4ff0-a68c-537e18ad4de4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.158430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:55.191523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8522-6775-4e85-813b-aae0c312e040 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.205314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.205973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.206316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.207566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.208669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.212805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.273866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.274817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.275365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.275858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.276773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.281165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 958/1939] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:55 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:55.282942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.283063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.317015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23324d-0f48-4973-a082-a67a33829508 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.325458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.326360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.326740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.327157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.328258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.330003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.332405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 982/1940] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:54 2022] POST /identity/v3/users => generated 292 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:55.338353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.348996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 959/1941] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:55.350512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.362379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3c37a-6807-4083-82cb-41de512d9b8d admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.362689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c95f56-4d5b-40a1-9035-f174bca0b3ea admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.369373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.370190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.370677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.371139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.372118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.374660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.374887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.376887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.377882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.378319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.386948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.389031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.392276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 960/1942] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:55.393454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.405998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1166e99-aa5d-490a-b993-dd6b6d19a1cd admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.411121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 983/1943] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:55.413180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb04a0-dddd-4d88-a9d3-fa1e1d862a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.413436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb04a0-dddd-4d88-a9d3-fa1e1d862a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.414766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb04a0-dddd-4d88-a9d3-fa1e1d862a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.415266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb04a0-dddd-4d88-a9d3-fa1e1d862a28 admin admin] PATH_INFO: `/v3/projects/5a4036efa2134d5dbeec212b7bfea0da/users/53cc5099a9374ba38350dca6787e47b6/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.418419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.433372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b631e1-ef44-483f-ae70-069b300feac0 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.434888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb04a0-dddd-4d88-a9d3-fa1e1d862a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4036efa2134d5dbeec212b7bfea0da, user_id=53cc5099a9374ba38350dca6787e47b6, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.438222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb04a0-dddd-4d88-a9d3-fa1e1d862a28 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.438353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.438839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.439154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.439438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.440088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.442152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.456917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 984/1944] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:55.462713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.465025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 961/1945] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:55 2022] PUT /identity/v3/projects/5a4036efa2134d5dbeec212b7bfea0da/users/53cc5099a9374ba38350dca6787e47b6/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:55.473359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc0f28-acc7-4b94-be63-6fd6cdfc8f48 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.475425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.475717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.475981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.504847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:55.512288 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] MFA Rules not processed for user `53cc5099a9374ba38350dca6787e47b6`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:55.513609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:55.560743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bc25f-5927-48fd-9b6d-9f5a69488eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.561671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bc25f-5927-48fd-9b6d-9f5a69488eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.562051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bc25f-5927-48fd-9b6d-9f5a69488eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.562459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bc25f-5927-48fd-9b6d-9f5a69488eb5 admin admin] PATH_INFO: `/v3/projects/3b251f2f9fa446be96cb8db35240422d/users/3ac2af515e6a49c79603dc0b569a978c/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.582489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bc25f-5927-48fd-9b6d-9f5a69488eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b251f2f9fa446be96cb8db35240422d, user_id=3ac2af515e6a49c79603dc0b569a978c, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.586162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bc25f-5927-48fd-9b6d-9f5a69488eb5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.598648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 962/1946] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:55.602910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 985/1947] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:55 2022] PUT /identity/v3/projects/3b251f2f9fa446be96cb8db35240422d/users/3ac2af515e6a49c79603dc0b569a978c/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:08:55.610604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.612135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:55.614108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.614393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.614661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.645071 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] MFA Rules not processed for user `3ac2af515e6a49c79603dc0b569a978c`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:55.646330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:55.682550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf5af8-2c50-4ab8-9987-6b5215938793 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.716198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:55.735192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 986/1948] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:08:55.746127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.747563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:55.762569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab38737-e72f-48c8-8eed-4f21f5013e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.763239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab38737-e72f-48c8-8eed-4f21f5013e2f service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.763848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab38737-e72f-48c8-8eed-4f21f5013e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.764133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab38737-e72f-48c8-8eed-4f21f5013e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.817488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfce94-e11e-43b1-9fa0-760531749fc8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.824063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.824688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.824972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.825249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.841511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.841694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab38737-e72f-48c8-8eed-4f21f5013e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.843224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab38737-e72f-48c8-8eed-4f21f5013e2f service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.843505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:55.855143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 987/1949] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:08:55.857370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 963/1950] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:55.880963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:55.957013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd3567-a611-40f1-866f-9e0b2ef1bc08 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:55.962734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:55.963457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:55.963688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:55.963963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] PATH_INFO: `/v3/users/3ac2af515e6a49c79603dc0b569a978c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:55.965106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac2af515e6a49c79603dc0b569a978c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:55.967116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:56.017245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac2af515e6a49c79603dc0b569a978c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.025839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac2af515e6a49c79603dc0b569a978c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.026474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac2af515e6a49c79603dc0b569a978c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.040600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac2af515e6a49c79603dc0b569a978c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.048744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac2af515e6a49c79603dc0b569a978c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.056216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 988/1951] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:08:55 2022] DELETE /identity/v3/users/3ac2af515e6a49c79603dc0b569a978c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:56.061227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715931-3a25-478a-9427-0bfe395844be None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.061528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715931-3a25-478a-9427-0bfe395844be None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.061822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715931-3a25-478a-9427-0bfe395844be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.082049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cc715931-3a25-478a-9427-0bfe395844be None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:56.083305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cc715931-3a25-478a-9427-0bfe395844be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:56.092976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:56.160746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 964/1952] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:56.171980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:56.173441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:56.205690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:56.247710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acda85-f770-4654-bfa5-d3d93eea779a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:56.251584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7c73-3953-4501-b6d5-d5d28307b083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:56.252195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7c73-3953-4501-b6d5-d5d28307b083 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.252757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7c73-3953-4501-b6d5-d5d28307b083 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.253037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7c73-3953-4501-b6d5-d5d28307b083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.263808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7c73-3953-4501-b6d5-d5d28307b083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:56.265433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7c73-3953-4501-b6d5-d5d28307b083 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:56.275848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 989/1953] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:08:56.572879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc715931-3a25-478a-9427-0bfe395844be None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:56.574605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cc715931-3a25-478a-9427-0bfe395844be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:56.589207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc715931-3a25-478a-9427-0bfe395844be None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:56.595273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:56.595918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.596197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.596597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] PATH_INFO: `/v3/projects/3b251f2f9fa446be96cb8db35240422d` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.597817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b251f2f9fa446be96cb8db35240422d)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:56.601347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:56.656754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b251f2f9fa446be96cb8db35240422d', 'request_id': 'req-410343b1-5622-4d4d-ab2a-0724c6a5d054'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.664071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b251f2f9fa446be96cb8db35240422d', 'request_id': 'req-410343b1-5622-4d4d-ab2a-0724c6a5d054'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.672547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b251f2f9fa446be96cb8db35240422d', 'request_id': 'req-410343b1-5622-4d4d-ab2a-0724c6a5d054'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.685129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] The token cache is being invalidate because project 3b251f2f9fa446be96cb8db35240422d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:08:56.686016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.689202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-410343b1-5622-4d4d-ab2a-0724c6a5d054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:08:56.689202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 965/1954] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:08:56 2022] DELETE /identity/v3/projects/3b251f2f9fa446be96cb8db35240422d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:56.692842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43873-6e63-4098-b425-889d0ab620fe None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.693130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43873-6e63-4098-b425-889d0ab620fe None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.693407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43873-6e63-4098-b425-889d0ab620fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.714358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d6c43873-6e63-4098-b425-889d0ab620fe None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:56.715619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d6c43873-6e63-4098-b425-889d0ab620fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:56.778307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 990/1955] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:56.786395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6008a-f778-4435-b251-2bb2e6f4cf4d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.786690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6008a-f778-4435-b251-2bb2e6f4cf4d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.786970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6008a-f778-4435-b251-2bb2e6f4cf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.807663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1cb6008a-f778-4435-b251-2bb2e6f4cf4d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:56.808880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1cb6008a-f778-4435-b251-2bb2e6f4cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:56.817680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 966/1956] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:56.822208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43873-6e63-4098-b425-889d0ab620fe None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:56.832222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43873-6e63-4098-b425-889d0ab620fe None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:56.838500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:56.839155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.839465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.839737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.840384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:08:56.840691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:56.842274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:56.855575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 991/1957] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:08:56.860448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6008a-f778-4435-b251-2bb2e6f4cf4d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:56.870070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6008a-f778-4435-b251-2bb2e6f4cf4d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:56.875004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:56.875652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.875934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.876210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.876864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:56.878941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:56.908004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 967/1958] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:56 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:56.912461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:56.923301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400a5a2-e939-43df-ab60-146f91463080 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:56.981957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:56.982573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:56.982853 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:56.983155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:56.983800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:56.985718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.012588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 992/1959] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:56 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:08:57.013788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.025067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bce5c-5d60-4c00-be58-c339253aa62f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.030220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.031221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.031612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.031951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.032680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.034348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.045276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 968/1960] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:57.049482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.060190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a4200-3ba4-4d41-a16d-31ab890723a6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.064720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601274d6-8086-48d5-bbb4-781583607127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.065319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601274d6-8086-48d5-bbb4-781583607127 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.065598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601274d6-8086-48d5-bbb4-781583607127 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.065872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601274d6-8086-48d5-bbb4-781583607127 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.066515 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601274d6-8086-48d5-bbb4-781583607127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.068145 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601274d6-8086-48d5-bbb4-781583607127 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.078663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 993/1961] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:57.082872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.093333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00b1ee-546e-4e70-befb-c4ac6c369148 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.098069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.098724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.099007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.099315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] PATH_INFO: `/v3/projects/248ee11b97cf4f95a09769f5a33366fe/users/019bb4a18f184a94877efd7176c59b4c/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.114903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=248ee11b97cf4f95a09769f5a33366fe, user_id=019bb4a18f184a94877efd7176c59b4c, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.118072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.133083 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 969/1962] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:57 2022] PUT /identity/v3/projects/248ee11b97cf4f95a09769f5a33366fe/users/019bb4a18f184a94877efd7176c59b4c/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:57.143833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4f5b7-314d-444c-810d-3fb9cfa96ef4 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.144159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4f5b7-314d-444c-810d-3fb9cfa96ef4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.144440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4f5b7-314d-444c-810d-3fb9cfa96ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.172928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c0c4f5b7-314d-444c-810d-3fb9cfa96ef4 None None] MFA Rules not processed for user `019bb4a18f184a94877efd7176c59b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:57.174134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c0c4f5b7-314d-444c-810d-3fb9cfa96ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:57.186708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:57.244706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 994/1963] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:57.249812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.289792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:57.322128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d0229-c8f5-46d6-a1ce-c7160c9233fc admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.327282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.327985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.328241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.328517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.329231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.331221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.360969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 970/1964] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:57 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:57.365578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4f5b7-314d-444c-810d-3fb9cfa96ef4 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.379522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4f5b7-314d-444c-810d-3fb9cfa96ef4 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.455121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.455747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.456033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.456311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.456944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.458884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.482424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 995/1965] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:57 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:57.486765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.497815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77633c-e9e3-4d46-afef-38b450c4a691 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.502512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.503049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.503368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.503644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.504294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.505899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.517229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 971/1966] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:57.521191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.531884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c2faf-4a5e-4552-9335-413807a0976b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.536553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40754e36-60de-4cc9-99c2-8ca4794bf409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.537179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40754e36-60de-4cc9-99c2-8ca4794bf409 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.537461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40754e36-60de-4cc9-99c2-8ca4794bf409 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.537737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40754e36-60de-4cc9-99c2-8ca4794bf409 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.538376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40754e36-60de-4cc9-99c2-8ca4794bf409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.540177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40754e36-60de-4cc9-99c2-8ca4794bf409 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.550074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 996/1967] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:57.556470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.566941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c7678-29cf-42f2-9557-1427ecf1fd3b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.572951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.573612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.573893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.574174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] PATH_INFO: `/v3/projects/3b78a3528de343078f14359bc604f8b4/users/4dd6680694854c0789b6a5779203f731/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.589934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b78a3528de343078f14359bc604f8b4, user_id=4dd6680694854c0789b6a5779203f731, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.593608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.611138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 972/1968] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:57 2022] PUT /identity/v3/projects/3b78a3528de343078f14359bc604f8b4/users/4dd6680694854c0789b6a5779203f731/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:57.621553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163851e7-1512-4435-99f7-9b6b60a4c04b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.621847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163851e7-1512-4435-99f7-9b6b60a4c04b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.622127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163851e7-1512-4435-99f7-9b6b60a4c04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.656652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-163851e7-1512-4435-99f7-9b6b60a4c04b None None] MFA Rules not processed for user `4dd6680694854c0789b6a5779203f731`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:57.657866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-163851e7-1512-4435-99f7-9b6b60a4c04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:57.722904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 997/1969] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:57.728286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.798961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6fd7-febe-4f8d-8188-276edba15d75 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.811415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.812458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.812817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.813182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.814027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.816009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.851013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 973/1970] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:08:57 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:57.855883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163851e7-1512-4435-99f7-9b6b60a4c04b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.869053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163851e7-1512-4435-99f7-9b6b60a4c04b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:57.893138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:57.955775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:57.956414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:57.956693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:57.957169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:57.957599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:57.959580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:57.990277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:57.990760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 998/1971] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:08:57 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:08:58.002088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ef80-0efa-49ab-b6a8-31c192764265 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:58.006699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:58.007335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:58.007698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:58.007980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:58.008655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:58.010353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:58.021458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 974/1972] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:08:58.026846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:58.037428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c2685-a496-4e09-bb57-fede93a5925d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:58.042852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:58.043555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:58.043833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:58.044104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] PATH_INFO: `/v3/projects/9dba5f6868344bfb9cb814326d6c6101/users/8e4569bf22ff4964ae4a306f02bd3fe5/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:58.059924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dba5f6868344bfb9cb814326d6c6101, user_id=8e4569bf22ff4964ae4a306f02bd3fe5, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:58.062609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:58.075534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 999/1973] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:58 2022] PUT /identity/v3/projects/9dba5f6868344bfb9cb814326d6c6101/users/8e4569bf22ff4964ae4a306f02bd3fe5/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:58.083495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:58.094919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dacda6-429f-4afc-8bfd-f1d3a5d0855a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:58.145009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:08:58.152229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219e803-1985-4369-af98-6a74b020df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:58.152951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219e803-1985-4369-af98-6a74b020df46 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:58.153436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219e803-1985-4369-af98-6a74b020df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:58.153869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219e803-1985-4369-af98-6a74b020df46 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:58.154677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219e803-1985-4369-af98-6a74b020df46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:58.156562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219e803-1985-4369-af98-6a74b020df46 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:58.166966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 975/1974] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:08:58.171311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:58.181972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92210a4-b5cf-47a3-9c0a-7b16839299c9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:58.186530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:58.187213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:58.187496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:58.187771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] PATH_INFO: `/v3/projects/9dba5f6868344bfb9cb814326d6c6101/users/8e4569bf22ff4964ae4a306f02bd3fe5/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:58.189707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dba5f6868344bfb9cb814326d6c6101, user_id=8e4569bf22ff4964ae4a306f02bd3fe5, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:58.192355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:58.202710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1000/1975] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:08:58 2022] PUT /identity/v3/projects/9dba5f6868344bfb9cb814326d6c6101/users/8e4569bf22ff4964ae4a306f02bd3fe5/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:08:58.213461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d4c4e-84a0-4d68-aa98-e32a945464d7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:58.213749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d4c4e-84a0-4d68-aa98-e32a945464d7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:58.214033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d4c4e-84a0-4d68-aa98-e32a945464d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:58.242264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-3c8d4c4e-84a0-4d68-aa98-e32a945464d7 None None] MFA Rules not processed for user `8e4569bf22ff4964ae4a306f02bd3fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:08:58.243536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3c8d4c4e-84a0-4d68-aa98-e32a945464d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:58.313059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 976/1976] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:08:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:08:58.324532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:08:58.326245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:08:58.383414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8837197-453a-42eb-9bb2-290f6d1986e9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:08:58.387141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:08:58.387896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:08:58.388028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:08:58.388302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:08:58.445332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:08:58.447236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:08:58.456753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1001/1977] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:08:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:02.071121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d4c4e-84a0-4d68-aa98-e32a945464d7 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:02.080831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d4c4e-84a0-4d68-aa98-e32a945464d7 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:02.084468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:02.085074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:02.085353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:02.086282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:02.144475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:02.146115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:02.157000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 977/1978] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:05.355015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:05.363686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22229383-2e95-4746-9c04-034203becf8b service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:05.367509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b139ec3-0ed0-4e8a-bfb3-c51df80d5255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:05.368120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b139ec3-0ed0-4e8a-bfb3-c51df80d5255 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:05.368406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b139ec3-0ed0-4e8a-bfb3-c51df80d5255 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:05.368681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b139ec3-0ed0-4e8a-bfb3-c51df80d5255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:05.377690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b139ec3-0ed0-4e8a-bfb3-c51df80d5255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:05.379150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b139ec3-0ed0-4e8a-bfb3-c51df80d5255 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:05.390367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1002/1979] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:09.516651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:09.586333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfc55a-bd65-46bb-8b90-4f8e8cfa3019 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:09.591629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:09.592313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:09.592618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:09.593308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] PATH_INFO: `/v3/users/019bb4a18f184a94877efd7176c59b4c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:09.594083 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=019bb4a18f184a94877efd7176c59b4c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:09.596170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:09.620479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:09.654419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019bb4a18f184a94877efd7176c59b4c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:09.668384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019bb4a18f184a94877efd7176c59b4c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:09.669252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019bb4a18f184a94877efd7176c59b4c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:09.669870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019bb4a18f184a94877efd7176c59b4c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:09.677956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019bb4a18f184a94877efd7176c59b4c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:09.692572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 978/1980] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:09 2022] DELETE /identity/v3/users/019bb4a18f184a94877efd7176c59b4c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:09.697662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:09.698149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:09.698149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:09.719619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:09.722273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:09.740348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:09.800435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1003/1981] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:09.811723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:09.813229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:09.910966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26361bd-7496-4d07-9a58-751e0882e98c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:09.915344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:09.916308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:09.916642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:09.916986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:09.928983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:09.931562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:09.942487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 979/1982] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:10.572485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:10.576081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:10.594709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96438d3a-91e3-4c92-ad15-b6464fb6fdb9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:10.600711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:10.601799 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:10.602124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:10.602442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] PATH_INFO: `/v3/projects/248ee11b97cf4f95a09769f5a33366fe` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:10.603976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=248ee11b97cf4f95a09769f5a33366fe)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:10.605935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:10.677791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248ee11b97cf4f95a09769f5a33366fe', 'request_id': 'req-7153f2e4-645b-42a4-a690-34f591b23b6c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.684841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248ee11b97cf4f95a09769f5a33366fe', 'request_id': 'req-7153f2e4-645b-42a4-a690-34f591b23b6c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.691596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248ee11b97cf4f95a09769f5a33366fe', 'request_id': 'req-7153f2e4-645b-42a4-a690-34f591b23b6c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.706138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] The token cache is being invalidate because project 248ee11b97cf4f95a09769f5a33366fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:10.706833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.707333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.708872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1004/1983] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:10 2022] DELETE /identity/v3/projects/248ee11b97cf4f95a09769f5a33366fe => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:10.713186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:10.724412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eceb410-29ac-49a9-bd29-3732932d36ae service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:10.818415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:10.820094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:10.821749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:10.822434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] PATH_INFO: `/v3/users/4dd6680694854c0789b6a5779203f731` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:10.826174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd6680694854c0789b6a5779203f731)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:10.828252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:10.890458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:10.903664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd6680694854c0789b6a5779203f731'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.922784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd6680694854c0789b6a5779203f731'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.924060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd6680694854c0789b6a5779203f731'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.924802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd6680694854c0789b6a5779203f731'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.934639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd6680694854c0789b6a5779203f731'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:10.956920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 980/1984] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:10 2022] DELETE /identity/v3/users/4dd6680694854c0789b6a5779203f731 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:11.471300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:11.472891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:11.487407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153f2e4-645b-42a4-a690-34f591b23b6c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:11.552757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:11.553517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:11.553870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:11.554278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] PATH_INFO: `/v3/projects/3b78a3528de343078f14359bc604f8b4` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:11.556272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b78a3528de343078f14359bc604f8b4)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:11.558240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:11.607818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:11.614387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b78a3528de343078f14359bc604f8b4', 'request_id': 'req-9509c29d-1cae-4669-9ce7-d63b31754bc2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.620246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:11.622547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:11.627489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b78a3528de343078f14359bc604f8b4', 'request_id': 'req-9509c29d-1cae-4669-9ce7-d63b31754bc2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.637815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ac758-32fb-4928-8e0f-c7b75fa494cb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:11.639955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b78a3528de343078f14359bc604f8b4', 'request_id': 'req-9509c29d-1cae-4669-9ce7-d63b31754bc2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.654650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] The token cache is being invalidate because project 3b78a3528de343078f14359bc604f8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:11.654942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.655780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.657582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1005/1985] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:11 2022] DELETE /identity/v3/projects/3b78a3528de343078f14359bc604f8b4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:11.661488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:11.663567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:11.707674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:11.709178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:11.709479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:11.709742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] PATH_INFO: `/v3/users/67f6989c3c4d4fe3a2cc3be83a5f5f0c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:11.711048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] RBAC: Authorizing `identity:delete_user(user_id=67f6989c3c4d4fe3a2cc3be83a5f5f0c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:11.713223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:11.733736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509c29d-1cae-4669-9ce7-d63b31754bc2 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:11.739139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:11.739743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:11.740058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:11.740316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] PATH_INFO: `/v3/users/8e4569bf22ff4964ae4a306f02bd3fe5` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:11.741483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e4569bf22ff4964ae4a306f02bd3fe5)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:11.744504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:11.771658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67f6989c3c4d4fe3a2cc3be83a5f5f0c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.786932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67f6989c3c4d4fe3a2cc3be83a5f5f0c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.787831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67f6989c3c4d4fe3a2cc3be83a5f5f0c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.788514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67f6989c3c4d4fe3a2cc3be83a5f5f0c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.796318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e4569bf22ff4964ae4a306f02bd3fe5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.799864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67f6989c3c4d4fe3a2cc3be83a5f5f0c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.807242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e4569bf22ff4964ae4a306f02bd3fe5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.809222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e4569bf22ff4964ae4a306f02bd3fe5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.817496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 981/1986] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:11 2022] DELETE /identity/v3/users/67f6989c3c4d4fe3a2cc3be83a5f5f0c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:11.826676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e4569bf22ff4964ae4a306f02bd3fe5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.834566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e4569bf22ff4964ae4a306f02bd3fe5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:11.841462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1006/1987] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:11 2022] DELETE /identity/v3/users/8e4569bf22ff4964ae4a306f02bd3fe5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:12.051202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.052999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.115464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:12.147956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321bb8e-5e7a-4a55-9f45-c6523ea7252c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.217997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.219682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.234129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.234778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.235063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.235369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] PATH_INFO: `/v3/projects/f530303e20704b88b87100476573d181` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.236544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f530303e20704b88b87100476573d181)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.238460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.291348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:12.311648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9755a-636f-4ac7-aa6d-8a11bce5ed6d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.311900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f530303e20704b88b87100476573d181', 'request_id': 'req-b0816244-8d86-4e74-a93f-77f6157d4daa'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.318646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f530303e20704b88b87100476573d181', 'request_id': 'req-b0816244-8d86-4e74-a93f-77f6157d4daa'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.328602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f530303e20704b88b87100476573d181', 'request_id': 'req-b0816244-8d86-4e74-a93f-77f6157d4daa'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.341137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] The token cache is being invalidate because project f530303e20704b88b87100476573d181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:12.341567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.342469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.346994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 982/1988] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:12 2022] DELETE /identity/v3/projects/f530303e20704b88b87100476573d181 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:12.348121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.349549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.385144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.385789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.386071 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.386346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] PATH_INFO: `/v3/projects/9dba5f6868344bfb9cb814326d6c6101` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.387533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dba5f6868344bfb9cb814326d6c6101)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.389502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.423868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816244-8d86-4e74-a93f-77f6157d4daa admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.438321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dba5f6868344bfb9cb814326d6c6101', 'request_id': 'req-c92acd24-9188-4990-be45-22b00e9ae4b2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.442829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:12.446446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dba5f6868344bfb9cb814326d6c6101', 'request_id': 'req-c92acd24-9188-4990-be45-22b00e9ae4b2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.452229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dba5f6868344bfb9cb814326d6c6101', 'request_id': 'req-c92acd24-9188-4990-be45-22b00e9ae4b2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.466216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] The token cache is being invalidate because project 9dba5f6868344bfb9cb814326d6c6101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:12.466519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.467253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c92acd24-9188-4990-be45-22b00e9ae4b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.468901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1007/1989] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:12 2022] DELETE /identity/v3/projects/9dba5f6868344bfb9cb814326d6c6101 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:12.473422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8081c05-a7f8-458e-a474-f76f6824acbc None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.473943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8081c05-a7f8-458e-a474-f76f6824acbc None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.474091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8081c05-a7f8-458e-a474-f76f6824acbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.486725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.487395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.487684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.487959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] PATH_INFO: `/v3/users/2f1c1e246b574f1e854cc6d2849b46bc` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.489091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1c1e246b574f1e854cc6d2849b46bc)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.491016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.496405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a8081c05-a7f8-458e-a474-f76f6824acbc None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:12.497631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a8081c05-a7f8-458e-a474-f76f6824acbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.506201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1008/1990] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:12.516720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b088-64f3-4b90-bf10-eb3d6e878d7b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.517123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b088-64f3-4b90-bf10-eb3d6e878d7b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.517787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2b088-64f3-4b90-bf10-eb3d6e878d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.542267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-82b2b088-64f3-4b90-bf10-eb3d6e878d7b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:12.543555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-82b2b088-64f3-4b90-bf10-eb3d6e878d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.544863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1c1e246b574f1e854cc6d2849b46bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.552288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1009/1991] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:12.558290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b088-64f3-4b90-bf10-eb3d6e878d7b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.558671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1c1e246b574f1e854cc6d2849b46bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.559435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1c1e246b574f1e854cc6d2849b46bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.560052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1c1e246b574f1e854cc6d2849b46bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.567474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1c1e246b574f1e854cc6d2849b46bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.567601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2b088-64f3-4b90-bf10-eb3d6e878d7b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.573858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.574581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.574880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.575204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.575877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:12.576187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.577796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.581526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 983/1992] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:12 2022] DELETE /identity/v3/users/2f1c1e246b574f1e854cc6d2849b46bc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:12.593475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1010/1993] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:09:12.594764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.596216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.666437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfed4c-1cc0-430d-ae28-e4b5d4b05efe admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.671716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.672353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.672633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.672905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.673577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.675512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.708490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 984/1994] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:12 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:12.710032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.720776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca27e7-57d5-469b-a59d-f17060ac9650 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.767171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.769037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.787379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.788155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.788449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.789371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.790370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.793109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.802866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:12.818294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1011/1995] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:12 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:12.823299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.825517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:12.835050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d73cd3-6853-49ed-9788-085e963ecb11 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.840361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.840967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.841244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.841519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.842164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.843792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.858320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1012/1996] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:12.859453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.863623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd68128-b720-4f72-928c-c60ea88568f4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.869154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.869797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.870073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.870341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] PATH_INFO: `/v3/projects/25eaf46184414a03951c8287c890619f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.870445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def06f7f-a455-4978-bdad-7e06ddf0d569 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.871549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25eaf46184414a03951c8287c890619f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.873543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.875254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.875889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.876198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.876477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.877164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.878814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.889507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1013/1997] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:12.893695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:12.903764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51e871-b75f-4515-951c-867ea846c1ed admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:12.909665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba33ac-5b60-4cf8-97e3-b60e521bbd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:12.910350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba33ac-5b60-4cf8-97e3-b60e521bbd7e admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.910658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba33ac-5b60-4cf8-97e3-b60e521bbd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.910934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba33ac-5b60-4cf8-97e3-b60e521bbd7e admin admin] PATH_INFO: `/v3/projects/0cc7f5fe616d47f9b58034ca0acd13e0/users/765e1fa5d82142fdbed266951a7650da/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.925567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25eaf46184414a03951c8287c890619f', 'request_id': 'req-c2f4d026-c4c4-4f61-9706-eadb791447f6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.927863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba33ac-5b60-4cf8-97e3-b60e521bbd7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc7f5fe616d47f9b58034ca0acd13e0, user_id=765e1fa5d82142fdbed266951a7650da, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:12.931280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba33ac-5b60-4cf8-97e3-b60e521bbd7e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:12.933186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25eaf46184414a03951c8287c890619f', 'request_id': 'req-c2f4d026-c4c4-4f61-9706-eadb791447f6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.940485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25eaf46184414a03951c8287c890619f', 'request_id': 'req-c2f4d026-c4c4-4f61-9706-eadb791447f6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.952089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1014/1998] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:12 2022] PUT /identity/v3/projects/0cc7f5fe616d47f9b58034ca0acd13e0/users/765e1fa5d82142fdbed266951a7650da/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:12.952408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] The token cache is being invalidate because project 25eaf46184414a03951c8287c890619f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:12.952700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.953423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c2f4d026-c4c4-4f61-9706-eadb791447f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:12.960143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 985/1999] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:12 2022] DELETE /identity/v3/projects/25eaf46184414a03951c8287c890619f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:12.962974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.963367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.963705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.966050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead911f-84b7-4745-9a1d-3a1d7e168c40 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:12.966338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead911f-84b7-4745-9a1d-3a1d7e168c40 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:12.966641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead911f-84b7-4745-9a1d-3a1d7e168c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:12.988297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5ead911f-84b7-4745-9a1d-3a1d7e168c40 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:12.989515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5ead911f-84b7-4745-9a1d-3a1d7e168c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:12.995303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] MFA Rules not processed for user `765e1fa5d82142fdbed266951a7650da`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:12.996517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:13.056751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 986/2000] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:13.058300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1aa9-395f-486c-a343-0b4a9438fab5 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.058593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1aa9-395f-486c-a343-0b4a9438fab5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.058871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1aa9-395f-486c-a343-0b4a9438fab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.064449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1015/2001] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:13.075841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.077262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:13.079683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-69cc1aa9-395f-486c-a343-0b4a9438fab5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:13.080861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-69cc1aa9-395f-486c-a343-0b4a9438fab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:13.091189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 987/2002] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:13.095236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc1aa9-395f-486c-a343-0b4a9438fab5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.106087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc1aa9-395f-486c-a343-0b4a9438fab5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.111194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.111874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.112221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.112693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.113393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:13.113588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.115194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.127461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 988/2003] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:09:13.131935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.146944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abc4da-744e-4cd6-aa2a-8920f9f5068d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.152019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.152637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.152917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.153191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.155204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ca50-7a6f-42e3-bd38-188f87ffee6e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.165312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.165610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.166039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.166330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.166610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.167329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.167454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.169796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.177460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1016/2004] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:13.210028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 989/2005] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:13 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:13.211769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.225746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005f094-748a-4ddd-b36e-155005a06fb5 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.278589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:13.315194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.315811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.316236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.316516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.317161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.319179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.341797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1017/2006] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:13 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:13.345989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.358310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74344681-f786-4152-b14a-6616c8eddf60 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.363275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.363963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.364244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.364555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.365392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.367129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.378690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 990/2007] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:13.382462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.395331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba26f71-6017-4565-a50f-5fd63a385d10 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.402007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10358fdc-ee10-46f9-8249-f4f1a4b3f2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.402617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358fdc-ee10-46f9-8249-f4f1a4b3f2ba admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.402896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358fdc-ee10-46f9-8249-f4f1a4b3f2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.403249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358fdc-ee10-46f9-8249-f4f1a4b3f2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.403892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10358fdc-ee10-46f9-8249-f4f1a4b3f2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.405687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10358fdc-ee10-46f9-8249-f4f1a4b3f2ba admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.425249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1018/2008] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:13.429750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.437944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:13.442542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf0d4e-a9df-484a-9fc0-435946ec1fd4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.454739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.457217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.459330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.459870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] PATH_INFO: `/v3/projects/0f594096296743c583bf3f8e0c1f5110/users/4c27047dbc2c40de89017865e41a0dc6/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.479622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f594096296743c583bf3f8e0c1f5110, user_id=4c27047dbc2c40de89017865e41a0dc6, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.482467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.499847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 991/2009] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:13 2022] PUT /identity/v3/projects/0f594096296743c583bf3f8e0c1f5110/users/4c27047dbc2c40de89017865e41a0dc6/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:13.511964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9ef9-5cc7-46b5-ba24-1ad0ae7ccfe4 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.512648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9ef9-5cc7-46b5-ba24-1ad0ae7ccfe4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.513122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9ef9-5cc7-46b5-ba24-1ad0ae7ccfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.559356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-516f9ef9-5cc7-46b5-ba24-1ad0ae7ccfe4 None None] MFA Rules not processed for user `4c27047dbc2c40de89017865e41a0dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:13.560580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-516f9ef9-5cc7-46b5-ba24-1ad0ae7ccfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:13.660396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1019/2010] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:13.674613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:13.737862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c2ae9-61cd-475b-936e-1fc14027f071 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:13.741836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:13.743361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:13.743723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:13.744001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:13.752674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:13.754115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:13.763496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 992/2011] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:15.138041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc7a22-7ae5-4d17-8558-5a6535c19a6d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:15.138600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc7a22-7ae5-4d17-8558-5a6535c19a6d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:15.138692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc7a22-7ae5-4d17-8558-5a6535c19a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:15.157902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-30dc7a22-7ae5-4d17-8558-5a6535c19a6d None None] MFA Rules not processed for user `4c27047dbc2c40de89017865e41a0dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:15.159143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-30dc7a22-7ae5-4d17-8558-5a6535c19a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:15.166705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1020/2012] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:19.480219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:19.482373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:19.656786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8062b-bf51-4429-9ae7-3826d21122b4 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:19.662861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:19.663606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:19.664507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:19.664904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] PATH_INFO: `/v3/users/53cc5099a9374ba38350dca6787e47b6` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:19.667138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53cc5099a9374ba38350dca6787e47b6)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:19.670776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:19.781212 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53cc5099a9374ba38350dca6787e47b6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:19.798298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53cc5099a9374ba38350dca6787e47b6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:19.809215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53cc5099a9374ba38350dca6787e47b6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:19.810418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53cc5099a9374ba38350dca6787e47b6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:19.829151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53cc5099a9374ba38350dca6787e47b6'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:19.860658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 993/2013] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:19 2022] DELETE /identity/v3/users/53cc5099a9374ba38350dca6787e47b6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:19.873692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:19.877068 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:19.877200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:19.908074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:19.909339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:19.988562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1021/2014] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:20.018201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:20.019703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:20.029531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:20.096237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3704d6-950d-4d67-a9cf-f0b93bd62929 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:20.100115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d95fce-cec7-49fc-8ab7-6520b4bf86cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:20.100801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d95fce-cec7-49fc-8ab7-6520b4bf86cb service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:20.101174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d95fce-cec7-49fc-8ab7-6520b4bf86cb service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:20.102046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d95fce-cec7-49fc-8ab7-6520b4bf86cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:20.112224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d95fce-cec7-49fc-8ab7-6520b4bf86cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:20.114009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d95fce-cec7-49fc-8ab7-6520b4bf86cb service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:20.125944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 994/2015] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:20.633636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:20.636776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:20.656773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3aae7-be75-401b-92e4-31f125757e11 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:20.670597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:20.671488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:20.671891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:20.672252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] PATH_INFO: `/v3/projects/5a4036efa2134d5dbeec212b7bfea0da` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:20.680872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4036efa2134d5dbeec212b7bfea0da)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:20.686161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:20.761785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:20.778532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4036efa2134d5dbeec212b7bfea0da', 'request_id': 'req-dbf600a0-d815-40c9-bee6-13438896edf0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:20.792080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4036efa2134d5dbeec212b7bfea0da', 'request_id': 'req-dbf600a0-d815-40c9-bee6-13438896edf0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:20.808475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4036efa2134d5dbeec212b7bfea0da', 'request_id': 'req-dbf600a0-d815-40c9-bee6-13438896edf0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:20.838461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] The token cache is being invalidate because project 5a4036efa2134d5dbeec212b7bfea0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:20.838785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:20.839616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dbf600a0-d815-40c9-bee6-13438896edf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:20.841520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1022/2016] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:20 2022] DELETE /identity/v3/projects/5a4036efa2134d5dbeec212b7bfea0da => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:20.846353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dd8b-a3f1-4f71-87c9-7e43b38ccd6b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:20.846877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dd8b-a3f1-4f71-87c9-7e43b38ccd6b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:20.846972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dd8b-a3f1-4f71-87c9-7e43b38ccd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:20.907293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-67e7dd8b-a3f1-4f71-87c9-7e43b38ccd6b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:20.908668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-67e7dd8b-a3f1-4f71-87c9-7e43b38ccd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:21.037795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 995/2017] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:21.046867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37695a27-1082-4e0a-bc33-9566cfaa6462 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.048459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37695a27-1082-4e0a-bc33-9566cfaa6462 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.049002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37695a27-1082-4e0a-bc33-9566cfaa6462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.085824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-37695a27-1082-4e0a-bc33-9566cfaa6462 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:21.088331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-37695a27-1082-4e0a-bc33-9566cfaa6462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:21.103943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7dd8b-a3f1-4f71-87c9-7e43b38ccd6b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.117505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1023/2018] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:21.123938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7dd8b-a3f1-4f71-87c9-7e43b38ccd6b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.134752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.136183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.136578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.137371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.140471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:21.140923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.145682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.171885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 996/2019] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:09:21.172266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37695a27-1082-4e0a-bc33-9566cfaa6462 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.186768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37695a27-1082-4e0a-bc33-9566cfaa6462 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.198722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.199258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.199538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.199817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.200540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.202859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.249891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1024/2020] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:21 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:21.255285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.268001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3886f2e-9ca4-4d03-b69e-82fc6e24ca49 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.398675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.399717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.400181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.400456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.401386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.405539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.442459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 997/2021] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:21 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:21.447418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.469314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8031159-c4d3-448e-96ac-0c05c5a13455 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.475844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.477348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.477968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.478356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.479403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.481191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.493496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1025/2022] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:21.497914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.513195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4352a48-7381-4477-bc85-576294349b23 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.517944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.518552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.518989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.519323 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.519971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.521948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.539826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 998/2023] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:21.540957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.557894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c54750-e0b7-4391-9be7-0ab78159e8eb admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.564423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98f460-9c20-48e4-8f71-8dde62d7ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.565114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98f460-9c20-48e4-8f71-8dde62d7ef8d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.565405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98f460-9c20-48e4-8f71-8dde62d7ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.565706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98f460-9c20-48e4-8f71-8dde62d7ef8d admin admin] PATH_INFO: `/v3/projects/8ed5c5779f13462d955099820f9d05b7/users/f65369cd04f744bea768b3154ff07536/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.581996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.583792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:21.592536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98f460-9c20-48e4-8f71-8dde62d7ef8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed5c5779f13462d955099820f9d05b7, user_id=f65369cd04f744bea768b3154ff07536, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.597397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98f460-9c20-48e4-8f71-8dde62d7ef8d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.605072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9aacf-afbd-4edf-8e2c-7b183590170a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.611400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.612171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.612549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.612917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] PATH_INFO: `/v3/users/cc580d348f154c4488a5f84640539655` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.614177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc580d348f154c4488a5f84640539655)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.614697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1026/2024] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:21 2022] PUT /identity/v3/projects/8ed5c5779f13462d955099820f9d05b7/users/f65369cd04f744bea768b3154ff07536/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:21.616540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.624018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:21.626191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cdf6b-ba02-4134-86dd-59e5da57b453 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.626445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cdf6b-ba02-4134-86dd-59e5da57b453 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.626719 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cdf6b-ba02-4134-86dd-59e5da57b453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.662605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-211cdf6b-ba02-4134-86dd-59e5da57b453 None None] MFA Rules not processed for user `f65369cd04f744bea768b3154ff07536`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:21.664608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-211cdf6b-ba02-4134-86dd-59e5da57b453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:21.678952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc580d348f154c4488a5f84640539655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:21.698038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc580d348f154c4488a5f84640539655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:21.699480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc580d348f154c4488a5f84640539655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:21.700421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc580d348f154c4488a5f84640539655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:21.709553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc580d348f154c4488a5f84640539655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:21.725248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 999/2025] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:21 2022] DELETE /identity/v3/users/cc580d348f154c4488a5f84640539655 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:21.753334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1027/2026] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:21.754820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.756392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:21.830327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eeb31f-7db3-49da-bba9-3cc4a5dc24dd admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:21.835545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:21.836757 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:21.837076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:21.837455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:21.838114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:21.840172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:21.868709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1000/2027] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:21 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:21.872522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cdf6b-ba02-4134-86dd-59e5da57b453 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:21.925370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:21.946836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cdf6b-ba02-4134-86dd-59e5da57b453 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.020192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.020816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.021100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.021370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.022218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.025231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.052462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1028/2028] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:21 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:22.058241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.076410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec36cf9-cb37-4ac4-b414-6f969c10e258 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.084068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.085146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.085614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.086688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.088098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.091527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.106745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1001/2029] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:22.111375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.122917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1e08b-8619-4e71-a17c-1a5d21bfc1cb admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.128606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762dc6f-33d6-4577-b139-658b85d013fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.129296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762dc6f-33d6-4577-b139-658b85d013fd admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.129784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762dc6f-33d6-4577-b139-658b85d013fd admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.130236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762dc6f-33d6-4577-b139-658b85d013fd admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.130993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762dc6f-33d6-4577-b139-658b85d013fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.132698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762dc6f-33d6-4577-b139-658b85d013fd admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.145138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1029/2030] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:22.150112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.160715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51051-f734-4f22-a4db-12e8d74f6853 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.166679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.167539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.167818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.168098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] PATH_INFO: `/v3/projects/87f83bccf74e45e3abf51089bcafbd52/users/417a5177627a448f9881681636ddc0a1/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.185671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f83bccf74e45e3abf51089bcafbd52, user_id=417a5177627a448f9881681636ddc0a1, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.188597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.202659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1002/2031] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:22 2022] PUT /identity/v3/projects/87f83bccf74e45e3abf51089bcafbd52/users/417a5177627a448f9881681636ddc0a1/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:22.214608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee847-ae4c-4d04-abf6-2e749a8c0911 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.215045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee847-ae4c-4d04-abf6-2e749a8c0911 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.215433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee847-ae4c-4d04-abf6-2e749a8c0911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.251501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b88ee847-ae4c-4d04-abf6-2e749a8c0911 None None] MFA Rules not processed for user `417a5177627a448f9881681636ddc0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:22.254407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b88ee847-ae4c-4d04-abf6-2e749a8c0911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:22.332114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.334479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:22.356242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1030/2032] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:22.360627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee847-ae4c-4d04-abf6-2e749a8c0911 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.390206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:22.443583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2bfa0-bd3d-46af-9bf5-cf7575221697 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.467111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.467111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.467111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.467111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] PATH_INFO: `/v3/projects/74f3f1161ef84f30bbd787e9361a0dac` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.467497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f3f1161ef84f30bbd787e9361a0dac)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.467497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.495352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee847-ae4c-4d04-abf6-2e749a8c0911 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.505952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.507090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.507393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.507957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.508984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.511377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.523869 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:22.525140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f3f1161ef84f30bbd787e9361a0dac', 'request_id': 'req-eae27214-8906-4343-a6c5-06fe038ab655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:22.532343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f3f1161ef84f30bbd787e9361a0dac', 'request_id': 'req-eae27214-8906-4343-a6c5-06fe038ab655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:22.541869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f3f1161ef84f30bbd787e9361a0dac', 'request_id': 'req-eae27214-8906-4343-a6c5-06fe038ab655'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:22.550713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1031/2033] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:22.553878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] The token cache is being invalidate because project 74f3f1161ef84f30bbd787e9361a0dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:22.554265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:22.554389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.555813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-eae27214-8906-4343-a6c5-06fe038ab655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:22.557786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1003/2034] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:22 2022] DELETE /identity/v3/projects/74f3f1161ef84f30bbd787e9361a0dac => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:22.562348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f519b2-9c0f-40df-b2ae-103f8785f046 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.562697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f519b2-9c0f-40df-b2ae-103f8785f046 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.563044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f519b2-9c0f-40df-b2ae-103f8785f046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.582618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c2f519b2-9c0f-40df-b2ae-103f8785f046 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:22.583873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c2f519b2-9c0f-40df-b2ae-103f8785f046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:22.619769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b93af8-d8e8-4fd7-b408-2fdaf2fcf6c6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.624508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.625112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.625393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.625666 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.626291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.628247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.650399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1032/2035] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:22.651772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1004/2036] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:22.655268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.656369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:22.659643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e4bf5-65d8-4fe6-ab9d-8ac799588703 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.659937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e4bf5-65d8-4fe6-ab9d-8ac799588703 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.660218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e4bf5-65d8-4fe6-ab9d-8ac799588703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.669653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb395ef0-8f80-4d4d-bece-09b6414452bc admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.675046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.675681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.675967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.676242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.676889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.678525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.681541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-3a5e4bf5-65d8-4fe6-ab9d-8ac799588703 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:22.682949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3a5e4bf5-65d8-4fe6-ab9d-8ac799588703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:22.690665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1033/2037] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:22.694451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1005/2038] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:22.694578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.695597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e4bf5-65d8-4fe6-ab9d-8ac799588703 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.704697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e4bf5-65d8-4fe6-ab9d-8ac799588703 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.706172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fed6e0-7928-49eb-8216-769e4685d98c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.709420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.710017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.710435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.710801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.711060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.711510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:22.711816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.712485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.712830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.713114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] PATH_INFO: `/v3/projects/5a50527857f54b37aeaeaf0d34d11c7a/users/d43652d12f35482a98aa4f263f954926/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.713442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.727122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1006/2039] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:09:22.731617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.733292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a50527857f54b37aeaeaf0d34d11c7a, user_id=d43652d12f35482a98aa4f263f954926, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.736004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.741597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203c5e9-b1fe-4c9a-bd04-f1c4f2a1ace5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.747393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.747994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.748274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.748654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.749281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.750562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1034/2040] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:22 2022] PUT /identity/v3/projects/5a50527857f54b37aeaeaf0d34d11c7a/users/d43652d12f35482a98aa4f263f954926/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:22.751182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.754523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.781817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1007/2041] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:22.786076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.811880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:22.844855 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd004fcf-134e-4747-a09e-6fd7b7fa7e3c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.859692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab25de-4b1e-46cb-a34b-61db64ab58cb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.864617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.865231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.865516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.865819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.866492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.868843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.894708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1008/2042] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:22.895452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:22.896489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.907057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.907689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.907966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.908068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898bd28-ef40-4371-bb0f-75ba1f789062 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.908235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.908878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.910459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.912911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.913513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.913918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.914749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.915455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.917070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.920965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1035/2043] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:22.924961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.927449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1009/2044] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:22.931402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.934578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc4656-4cf3-4e74-b1a1-c663a675c0e9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.939240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.939825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.940105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.940377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] PATH_INFO: `/v3/projects/5a50527857f54b37aeaeaf0d34d11c7a/users/d43652d12f35482a98aa4f263f954926/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.941311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bad39-3389-4a09-a862-cc32eb3123e0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:22.942248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a50527857f54b37aeaeaf0d34d11c7a, user_id=d43652d12f35482a98aa4f263f954926, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.944896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.946001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbe469-1feb-4fd6-924c-5c1fd2a749a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:22.946588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe469-1feb-4fd6-924c-5c1fd2a749a3 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.946871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe469-1feb-4fd6-924c-5c1fd2a749a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.947178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe469-1feb-4fd6-924c-5c1fd2a749a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.947930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbe469-1feb-4fd6-924c-5c1fd2a749a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:22.949506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbe469-1feb-4fd6-924c-5c1fd2a749a3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:22.955406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1036/2045] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:22 2022] PUT /identity/v3/projects/5a50527857f54b37aeaeaf0d34d11c7a/users/d43652d12f35482a98aa4f263f954926/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:22.960109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1010/2046] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:22.964851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:22.966129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01646-f28f-4282-bea2-a78851d6affa None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:22.966415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01646-f28f-4282-bea2-a78851d6affa None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:22.966693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01646-f28f-4282-bea2-a78851d6affa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:22.995213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7cd01646-f28f-4282-bea2-a78851d6affa None None] MFA Rules not processed for user `d43652d12f35482a98aa4f263f954926`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:22.996485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7cd01646-f28f-4282-bea2-a78851d6affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:23.028140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bcb22-c387-4c6b-a596-32fd33312eae admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:23.033173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:23.033829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:23.034112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:23.034389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] PATH_INFO: `/v3/projects/95fbab83c1c04b879461b820b1c5827f/users/849c01285b1a498b8418137a96a121a4/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:23.050256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95fbab83c1c04b879461b820b1c5827f, user_id=849c01285b1a498b8418137a96a121a4, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:23.053612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:23.067274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1011/2047] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:23.068110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1037/2048] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:22 2022] PUT /identity/v3/projects/95fbab83c1c04b879461b820b1c5827f/users/849c01285b1a498b8418137a96a121a4/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:23.079202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:23.079377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:23.079517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:23.079811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:23.081045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:23.110565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] MFA Rules not processed for user `849c01285b1a498b8418137a96a121a4`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:23.111928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:23.140884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484e2d5-6a83-4c1d-b259-83c2582a80d1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:23.144608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:23.145447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:23.145729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:23.146006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:23.150177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:23.189940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1012/2049] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:23.202531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:23.204193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:23.212638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:23.219692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:23.221274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:23.231670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1038/2050] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:23.303002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264115bf-58e7-4b74-865d-eaeecf6b146c None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:23.310428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26a2c-9155-40cc-918f-0a0c45ccd4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:23.311098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26a2c-9155-40cc-918f-0a0c45ccd4d2 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:23.311390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26a2c-9155-40cc-918f-0a0c45ccd4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:23.311662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26a2c-9155-40cc-918f-0a0c45ccd4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:23.321047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db26a2c-9155-40cc-918f-0a0c45ccd4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:23.323038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db26a2c-9155-40cc-918f-0a0c45ccd4d2 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:23.331840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1013/2051] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:24.336841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:24.338903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:24.353644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:24.412874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71df6c-d3a4-4e1c-b595-8f4b08071e62 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:24.420647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:24.420647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:24.420647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:24.420647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] PATH_INFO: `/v3/users/765e1fa5d82142fdbed266951a7650da` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:24.421017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=765e1fa5d82142fdbed266951a7650da)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:24.422755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:24.470866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765e1fa5d82142fdbed266951a7650da'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:24.478516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765e1fa5d82142fdbed266951a7650da'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:24.479271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765e1fa5d82142fdbed266951a7650da'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:24.492926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765e1fa5d82142fdbed266951a7650da'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:24.500440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765e1fa5d82142fdbed266951a7650da'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:24.506700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1039/2052] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:24 2022] DELETE /identity/v3/users/765e1fa5d82142fdbed266951a7650da => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:24.511732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:24.512200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:24.512297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:24.532264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:24.533470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:24.603376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1014/2053] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:24.614050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:24.615597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:24.686024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780a6d-fe62-478b-99ec-06dc53fee6ea admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:24.690346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:24.691242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:24.691669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:24.692209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:24.705612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:24.707372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:24.719753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1040/2054] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:24.788070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:24.789776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:24.920411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba8d69-e5f3-4c3c-b12f-11af34ed409d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:24.925935 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7640f-c47c-4e62-b89a-685df793279b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:24.926548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7640f-c47c-4e62-b89a-685df793279b service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:24.926831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7640f-c47c-4e62-b89a-685df793279b service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:24.927133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7640f-c47c-4e62-b89a-685df793279b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:24.929037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7ab7640f-c47c-4e62-b89a-685df793279b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:24.999307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:25.021028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7640f-c47c-4e62-b89a-685df793279b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.024515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7640f-c47c-4e62-b89a-685df793279b service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.036221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1015/2055] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:25.275851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.277378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:25.302694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2748d39-a4db-4879-afc0-08c553b8ab78 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.318720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.320002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.320387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.320725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] PATH_INFO: `/v3/projects/0cc7f5fe616d47f9b58034ca0acd13e0` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.321957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc7f5fe616d47f9b58034ca0acd13e0)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.324082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.361112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:25.377714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc7f5fe616d47f9b58034ca0acd13e0', 'request_id': 'req-1ef09ed3-80b9-4e28-8276-02009a1046fd'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:25.384406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc7f5fe616d47f9b58034ca0acd13e0', 'request_id': 'req-1ef09ed3-80b9-4e28-8276-02009a1046fd'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:25.389971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc7f5fe616d47f9b58034ca0acd13e0', 'request_id': 'req-1ef09ed3-80b9-4e28-8276-02009a1046fd'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:25.404131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] The token cache is being invalidate because project 0cc7f5fe616d47f9b58034ca0acd13e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:25.404419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:25.405223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1ef09ed3-80b9-4e28-8276-02009a1046fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:25.407222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1041/2056] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:25 2022] DELETE /identity/v3/projects/0cc7f5fe616d47f9b58034ca0acd13e0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:25.412095 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc33b3-acbd-4511-bc84-58e99b5f7798 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.412396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc33b3-acbd-4511-bc84-58e99b5f7798 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.412701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc33b3-acbd-4511-bc84-58e99b5f7798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.435256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b8fc33b3-acbd-4511-bc84-58e99b5f7798 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:25.436415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b8fc33b3-acbd-4511-bc84-58e99b5f7798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:25.505530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1016/2057] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:25.515369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c240-86ae-4cfe-a94d-9994d065a4e1 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.515658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c240-86ae-4cfe-a94d-9994d065a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.515960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c240-86ae-4cfe-a94d-9994d065a4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.538768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-6784c240-86ae-4cfe-a94d-9994d065a4e1 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:25.540087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6784c240-86ae-4cfe-a94d-9994d065a4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:25.548226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1042/2058] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:25.552746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc33b3-acbd-4511-bc84-58e99b5f7798 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.563592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc33b3-acbd-4511-bc84-58e99b5f7798 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.568264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.568897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.569291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.569570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.571153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:25.571651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.573265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.586795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1017/2059] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:09:25.591684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c240-86ae-4cfe-a94d-9994d065a4e1 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.605579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c240-86ae-4cfe-a94d-9994d065a4e1 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.613037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.613982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.614468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.614879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.615938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.620003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.650601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1043/2060] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:25 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:25.658794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.665599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84374-5cc9-47ed-a282-fa5a2dd0b353 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.727739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.728417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.728747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.729028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.729711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.731793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.744472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.746518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:25.758946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1018/2061] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:25 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:25.760018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.769685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadacce4-ea7e-46a3-ba60-c40c1890162f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.774995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.775647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.775951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.776209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.776925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.778508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.791121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1019/2062] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:25.793628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.804351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e32cdb-b0ab-4217-af5d-084bfbcfc2b3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.810016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.810835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.811249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.812135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.812254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.813780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.815833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb8442-53b9-4c7d-acd7-7e422ecb0474 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.820038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.820936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.821207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.822125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.824125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:25.829600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1020/2063] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:25.833747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:25.848423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9d4e2-1007-45b4-8845-4d05bd598f19 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:25.853571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387a6b0-a3df-4ab0-bc9c-feca07257312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:25.854235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387a6b0-a3df-4ab0-bc9c-feca07257312 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.854513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387a6b0-a3df-4ab0-bc9c-feca07257312 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.854885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387a6b0-a3df-4ab0-bc9c-feca07257312 admin admin] PATH_INFO: `/v3/projects/778d0dad05594da5bb3cb0e2efcb2af5/users/985dfcc7246c45b5b470e07957cd2f49/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.873238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387a6b0-a3df-4ab0-bc9c-feca07257312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778d0dad05594da5bb3cb0e2efcb2af5, user_id=985dfcc7246c45b5b470e07957cd2f49, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.876937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387a6b0-a3df-4ab0-bc9c-feca07257312 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.893067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1021/2064] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:25 2022] PUT /identity/v3/projects/778d0dad05594da5bb3cb0e2efcb2af5/users/985dfcc7246c45b5b470e07957cd2f49/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:25.896796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:25.898041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2947-6674-413b-a592-0e462f0c70d9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:25.898282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:25.898524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2947-6674-413b-a592-0e462f0c70d9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:25.898878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b2947-6674-413b-a592-0e462f0c70d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:25.909883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1044/2065] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:25.921261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a51b2947-6674-413b-a592-0e462f0c70d9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:25.922736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a51b2947-6674-413b-a592-0e462f0c70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:25.933368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:25.997108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1022/2066] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:26.009598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:26.011386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:26.058648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:26.076425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec390960-86f2-4e32-879c-218d091d47de service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:26.125750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:26.126353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:26.126629 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:26.126896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:26.135829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:26.137260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:26.146597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1045/2067] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:26.863192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22696aeb-c56a-4405-9ea1-149351f9a405 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:26.863667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22696aeb-c56a-4405-9ea1-149351f9a405 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:26.863758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22696aeb-c56a-4405-9ea1-149351f9a405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:26.894100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-22696aeb-c56a-4405-9ea1-149351f9a405 None None] MFA Rules not processed for user `985dfcc7246c45b5b470e07957cd2f49`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:26.895360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-22696aeb-c56a-4405-9ea1-149351f9a405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:26.964028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1023/2068] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:26.974628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:26.976212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:27.036661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fd333-c53e-4735-9087-aff589dc6bc5 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:27.041690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:27.042310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:27.042595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:27.042873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:27.051992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:27.053428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:27.063753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1046/2069] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:29.496815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476210c-be00-4c23-a872-df336633a379 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:29.497309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476210c-be00-4c23-a872-df336633a379 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:29.497309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476210c-be00-4c23-a872-df336633a379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:29.522985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c476210c-be00-4c23-a872-df336633a379 None None] MFA Rules not processed for user `985dfcc7246c45b5b470e07957cd2f49`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:29.524982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c476210c-be00-4c23-a872-df336633a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:29.532435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1024/2070] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:31.972576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:31.973980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:31.988380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f705-c287-4712-94f0-eede3d69f13f service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:31.993674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:31.994325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:31.994609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:31.994904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] PATH_INFO: `/v3/users/f65369cd04f744bea768b3154ff07536` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:31.996276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65369cd04f744bea768b3154ff07536)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:31.998282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:32.013151 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476210c-be00-4c23-a872-df336633a379 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:32.014563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c476210c-be00-4c23-a872-df336633a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:32.028017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476210c-be00-4c23-a872-df336633a379 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:32.033566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:32.034224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:32.034479 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:32.034751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] PATH_INFO: `/v3/users/849c01285b1a498b8418137a96a121a4` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:32.035923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849c01285b1a498b8418137a96a121a4)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:32.038051 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:32.053178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65369cd04f744bea768b3154ff07536'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.061227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65369cd04f744bea768b3154ff07536'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.061978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65369cd04f744bea768b3154ff07536'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.075643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65369cd04f744bea768b3154ff07536'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.082577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a497e216-8f42-4feb-9ea9-42bbaf7fe3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65369cd04f744bea768b3154ff07536'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.089108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1047/2071] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:31 2022] DELETE /identity/v3/users/f65369cd04f744bea768b3154ff07536 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:32.089268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849c01285b1a498b8418137a96a121a4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.093532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:32.093824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:32.094113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:32.108813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849c01285b1a498b8418137a96a121a4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.109588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849c01285b1a498b8418137a96a121a4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.110257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849c01285b1a498b8418137a96a121a4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.114481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:32.115741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:32.118605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-e1bd2893-cc72-4e83-b411-30ed443d76b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849c01285b1a498b8418137a96a121a4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.123808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:32.139245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1025/2072] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:32 2022] DELETE /identity/v3/users/849c01285b1a498b8418137a96a121a4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:32.143966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:32.144494 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:32.144817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:32.166036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:32.167593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:32.174655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:32.188919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1048/2073] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:32.198895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:32.200347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:32.243934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1026/2074] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:32.256543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:32.258753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:32.278282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881bd6a-1710-4bb6-9afd-5dae5f3cda2e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:32.330424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f48880-59df-4c3a-8e7f-174d16573260 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:32.334233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:32.334960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:32.335368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:32.335754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:32.347767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:32.348951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:32.349575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:32.350477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:32.350600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:32.350600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:32.360734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:32.362195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:32.363420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1027/2075] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:32.373709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1049/2076] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:32.874592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:32.876119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:32.890203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a6dd1-81cb-4281-b84c-547d6d2af59a service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:32.895657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:32.896140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:32.896423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:32.896701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] PATH_INFO: `/v3/projects/95fbab83c1c04b879461b820b1c5827f` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:32.897999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95fbab83c1c04b879461b820b1c5827f)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:32.900609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:32.906414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:32.910089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:32.930388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b6175-e32c-4392-ab3d-012d430bae36 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:32.937014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:32.937789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:32.938085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:32.938383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] PATH_INFO: `/v3/projects/8ed5c5779f13462d955099820f9d05b7` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:32.940844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed5c5779f13462d955099820f9d05b7)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:32.942851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:32.966666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95fbab83c1c04b879461b820b1c5827f', 'request_id': 'req-5aad5013-71d0-4a0e-8de9-e472f97c7428'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.973245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95fbab83c1c04b879461b820b1c5827f', 'request_id': 'req-5aad5013-71d0-4a0e-8de9-e472f97c7428'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.980544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95fbab83c1c04b879461b820b1c5827f', 'request_id': 'req-5aad5013-71d0-4a0e-8de9-e472f97c7428'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.992998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] The token cache is being invalidate because project 95fbab83c1c04b879461b820b1c5827f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:32.993339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.993987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5aad5013-71d0-4a0e-8de9-e472f97c7428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.995025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed5c5779f13462d955099820f9d05b7', 'request_id': 'req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:32.995898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1028/2077] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:32 2022] DELETE /identity/v3/projects/95fbab83c1c04b879461b820b1c5827f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:33.000313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa8637-8248-4ac2-acc7-90eddd351354 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.000594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa8637-8248-4ac2-acc7-90eddd351354 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.000861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa8637-8248-4ac2-acc7-90eddd351354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.002417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed5c5779f13462d955099820f9d05b7', 'request_id': 'req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.010083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed5c5779f13462d955099820f9d05b7', 'request_id': 'req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.013913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:33.024012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-fdaa8637-8248-4ac2-acc7-90eddd351354 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:33.025808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fdaa8637-8248-4ac2-acc7-90eddd351354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:33.029198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] The token cache is being invalidate because project 8ed5c5779f13462d955099820f9d05b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:33.029554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.030369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.032174 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1050/2078] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:32 2022] DELETE /identity/v3/projects/8ed5c5779f13462d955099820f9d05b7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:33.036807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.038650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:33.064166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:33.094687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1029/2079] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:33.104069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbae27-3906-4120-9cde-f865aab67782 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.104408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbae27-3906-4120-9cde-f865aab67782 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.104683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbae27-3906-4120-9cde-f865aab67782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.108570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6d5f7-6fc7-4aac-a237-b9ede9853094 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.113158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.113775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.114064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.114331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] PATH_INFO: `/v3/users/417a5177627a448f9881681636ddc0a1` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.115506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=417a5177627a448f9881681636ddc0a1)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.118127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.126108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8cdbae27-3906-4120-9cde-f865aab67782 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:33.127610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8cdbae27-3906-4120-9cde-f865aab67782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:33.139210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1030/2080] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:33.143218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbae27-3906-4120-9cde-f865aab67782 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.154692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbae27-3906-4120-9cde-f865aab67782 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.160540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.161244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.161470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.161748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.162523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:33.162917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.166424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.171168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417a5177627a448f9881681636ddc0a1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.182744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1031/2081] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:09:33.185412 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417a5177627a448f9881681636ddc0a1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.186117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417a5177627a448f9881681636ddc0a1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.187680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.199844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bec758-c90b-442e-8576-58cc332ab0d3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.200315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417a5177627a448f9881681636ddc0a1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.204896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.205583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.205872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.206193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.206891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.207665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417a5177627a448f9881681636ddc0a1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:33.208858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.215997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1051/2082] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:33 2022] DELETE /identity/v3/users/417a5177627a448f9881681636ddc0a1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:33.257197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1032/2083] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:33 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:33.260338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.262738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:33.364306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447e93b-c208-4dba-a7c1-d23dddb0c69c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.390806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:33.478361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.479952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.480354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.480775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.481597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.484734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.514878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1052/2084] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:33 2022] POST /identity/v3/users => generated 282 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:33.518925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.529726 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40716f9-6bd9-4fd0-a3a2-304180fe440a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.534646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.535295 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.535576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.535853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.536507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.538120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.549750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1033/2085] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:33.553794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.566719 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1d10-8001-4a0c-982e-2fc82c5cdfb5 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.574387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6932e5-bf76-4c21-bab8-691ed9f49686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.575201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6932e5-bf76-4c21-bab8-691ed9f49686 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.575557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6932e5-bf76-4c21-bab8-691ed9f49686 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.575920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6932e5-bf76-4c21-bab8-691ed9f49686 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.576636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6932e5-bf76-4c21-bab8-691ed9f49686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.578297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6932e5-bf76-4c21-bab8-691ed9f49686 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.595367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1053/2086] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:33.599057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.611194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df331-265a-49f4-a257-f359d4511281 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.617864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.618687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.619065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.619602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] PATH_INFO: `/v3/projects/f9c3a16a2dd24da0a7d5adde8c284fbf/users/e9820162f8d74e1ab1865d80d5dc2f5d/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.638701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c3a16a2dd24da0a7d5adde8c284fbf, user_id=e9820162f8d74e1ab1865d80d5dc2f5d, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.641632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.657321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1034/2087] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:33 2022] PUT /identity/v3/projects/f9c3a16a2dd24da0a7d5adde8c284fbf/users/e9820162f8d74e1ab1865d80d5dc2f5d/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:33.668817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.669179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.669569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.700704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] MFA Rules not processed for user `e9820162f8d74e1ab1865d80d5dc2f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:33.702118 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:33.778444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1054/2088] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:33.783899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.873638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:33.875228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:33.884698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39407a15-80f6-4c53-b7bd-d18d5ea0ded4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.890143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.890770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.891063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.891364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.892056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.893938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.902858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415ed90-3d9a-4d21-b4c3-577713fd904e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:33.914222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:33.914944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:33.915553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:33.915553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] PATH_INFO: `/v3/projects/87f83bccf74e45e3abf51089bcafbd52` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:33.916728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f83bccf74e45e3abf51089bcafbd52)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:33.918732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:33.985956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1035/2089] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:33 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:33.990632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.031527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f83bccf74e45e3abf51089bcafbd52', 'request_id': 'req-c99a9659-9d73-4f68-9a61-f7977e70e538'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.043871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f83bccf74e45e3abf51089bcafbd52', 'request_id': 'req-c99a9659-9d73-4f68-9a61-f7977e70e538'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.059148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f83bccf74e45e3abf51089bcafbd52', 'request_id': 'req-c99a9659-9d73-4f68-9a61-f7977e70e538'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.074020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:34.088107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] The token cache is being invalidate because project 87f83bccf74e45e3abf51089bcafbd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:34.088872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.090712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.093551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1055/2090] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:33 2022] DELETE /identity/v3/projects/87f83bccf74e45e3abf51089bcafbd52 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:34.099849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25386499-6448-4696-a74e-6fb213ce75fa admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.107322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.112514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:34.136291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:34.222018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:34.222662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.223054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.223563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.224366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:34.228075 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:34.263535 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1036/2091] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:33 2022] POST /identity/v3/users => generated 289 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:34.267976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.269660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a9659-9d73-4f68-9a61-f7977e70e538 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.270164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:34.276954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:34.277708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.278083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.278453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] PATH_INFO: `/v3/users/d43652d12f35482a98aa4f263f954926` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.279735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43652d12f35482a98aa4f263f954926)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:34.281792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:34.287807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9aa6-6b40-430a-9c7c-239b0f27a8c5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.293325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:34.293950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.294234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.294659 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.295566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:34.297192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:34.308877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1037/2092] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:34.313819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.326383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5aa7-2edf-4544-9e7e-f858528d18e3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.331597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:34.332283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.332580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.332864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] PATH_INFO: `/v3/projects/7eb85ae214e647f690b1b8493a83ee35/users/7f4ad20324134965b110943230be35e0/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.341001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43652d12f35482a98aa4f263f954926'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.350152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43652d12f35482a98aa4f263f954926'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.350958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb85ae214e647f690b1b8493a83ee35, user_id=7f4ad20324134965b110943230be35e0, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:34.351054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43652d12f35482a98aa4f263f954926'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.353843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:34.367022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43652d12f35482a98aa4f263f954926'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.375542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1038/2093] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:34 2022] PUT /identity/v3/projects/7eb85ae214e647f690b1b8493a83ee35/users/7f4ad20324134965b110943230be35e0/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:34.376521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43652d12f35482a98aa4f263f954926'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.379374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.381150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:34.385645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1056/2094] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:34 2022] DELETE /identity/v3/users/d43652d12f35482a98aa4f263f954926 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:34.463000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1049ed-f7fe-4e4d-8405-44c30d975522 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.529768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:34.538651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62019d-3c8b-4df2-820e-13c894e92cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:34.541378 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62019d-3c8b-4df2-820e-13c894e92cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.541967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62019d-3c8b-4df2-820e-13c894e92cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.542415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62019d-3c8b-4df2-820e-13c894e92cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.543702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62019d-3c8b-4df2-820e-13c894e92cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:34.545522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62019d-3c8b-4df2-820e-13c894e92cf1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:34.561240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1039/2095] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:34.565645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.607398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75118-988e-4694-b961-4471bdd6edf3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.620510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:34.621216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.621476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.621750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] PATH_INFO: `/v3/projects/7eb85ae214e647f690b1b8493a83ee35/users/7f4ad20324134965b110943230be35e0/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.626688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb85ae214e647f690b1b8493a83ee35, user_id=7f4ad20324134965b110943230be35e0, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:34.629408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:34.643014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1057/2096] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:34 2022] PUT /identity/v3/projects/7eb85ae214e647f690b1b8493a83ee35/users/7f4ad20324134965b110943230be35e0/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:34.654648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.654889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.655176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.688402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] MFA Rules not processed for user `7f4ad20324134965b110943230be35e0`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:34.689960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:34.734244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.738057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:34.794626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1040/2097] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:34.809646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:34.812129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:34.861594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a284f-e380-4724-bba4-b3e7e80c290d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.868429 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:34.869151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:34.869442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:34.870141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] PATH_INFO: `/v3/projects/5a50527857f54b37aeaeaf0d34d11c7a` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:34.872498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a50527857f54b37aeaeaf0d34d11c7a)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:34.878216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:34.913856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:34.929737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f7466-c5fd-4938-9b14-e6d96eb292a7 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:34.950063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a50527857f54b37aeaeaf0d34d11c7a', 'request_id': 'req-c5483d1f-cc51-46d3-9e62-3026538e37b2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.957718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a50527857f54b37aeaeaf0d34d11c7a', 'request_id': 'req-c5483d1f-cc51-46d3-9e62-3026538e37b2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.965466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a50527857f54b37aeaeaf0d34d11c7a', 'request_id': 'req-c5483d1f-cc51-46d3-9e62-3026538e37b2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.988930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] The token cache is being invalidate because project 5a50527857f54b37aeaeaf0d34d11c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:34.989330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.990458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c5483d1f-cc51-46d3-9e62-3026538e37b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:34.993562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1058/2098] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:34 2022] DELETE /identity/v3/projects/5a50527857f54b37aeaeaf0d34d11c7a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:34.999958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d79525-36a0-44b8-b598-da52926750da None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.000417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d79525-36a0-44b8-b598-da52926750da None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.001178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d79525-36a0-44b8-b598-da52926750da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.030989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-50d79525-36a0-44b8-b598-da52926750da None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:35.033092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-50d79525-36a0-44b8-b598-da52926750da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:35.058509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.059122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.059411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.059690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.062898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:35.119329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1059/2099] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:35.126890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510caf3-a74c-4416-a9eb-bb8a481fc5fe None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.127207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510caf3-a74c-4416-a9eb-bb8a481fc5fe None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.127506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510caf3-a74c-4416-a9eb-bb8a481fc5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.128557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:35.150741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.152306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0510caf3-a74c-4416-a9eb-bb8a481fc5fe None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:35.153232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.153535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0510caf3-a74c-4416-a9eb-bb8a481fc5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:35.163747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1060/2100] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:35.166897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510caf3-a74c-4416-a9eb-bb8a481fc5fe None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.169110 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1041/2101] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:35.177061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510caf3-a74c-4416-a9eb-bb8a481fc5fe None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.182276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.182894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.183204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.183475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.184126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:35.184428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.186067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.223121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1061/2102] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:09:35.228734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.248964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c265ba-9f49-42bf-beee-09ee602e584b service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.255039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.255967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.256867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.257058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.259662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.262538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.298356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1042/2103] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:35 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:35.300417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.302345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:35.302468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.318538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c87278-ab14-46ff-9e6a-08b8f25b250b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.379336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.380511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.380511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.380511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.383345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:35.393149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7028099-28df-4adf-b29a-071c32f1da94 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.398214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.399821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.400133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.400408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.401056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.403228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.437723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:35.451842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1043/2104] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:35 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:35.456146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.467047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.469792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.475346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92229e-ea63-4194-8484-62c6d90e0862 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.480729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.481412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.481700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.481973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.482646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.484270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.502548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1044/2105] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:35.502944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1062/2106] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:35.503039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.519989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b99d-c31d-4752-b24e-a1cb8f474bd5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.528058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f2e87-9d84-4825-a109-ace15af7d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.528696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2e87-9d84-4825-a109-ace15af7d7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.528981 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2e87-9d84-4825-a109-ace15af7d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.529258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2e87-9d84-4825-a109-ace15af7d7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.529995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f2e87-9d84-4825-a109-ace15af7d7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.531599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f2e87-9d84-4825-a109-ace15af7d7a5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.552350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1045/2107] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:35.555616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.567501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16791bb0-abf5-4419-ac29-98369440d7a3 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.573878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.574818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.575214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.575529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] PATH_INFO: `/v3/projects/6a4d940594f542808602c7260993f85f/users/8f1c0b2189a041fdb8dbe2a0290ffac8/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.592962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4d940594f542808602c7260993f85f, user_id=8f1c0b2189a041fdb8dbe2a0290ffac8, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.596020 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.615137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1063/2108] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:35 2022] PUT /identity/v3/projects/6a4d940594f542808602c7260993f85f/users/8f1c0b2189a041fdb8dbe2a0290ffac8/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:35.620928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d64e4-2d4b-4b34-a41e-9453f7a4dd5e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.621211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d64e4-2d4b-4b34-a41e-9453f7a4dd5e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.621693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d64e4-2d4b-4b34-a41e-9453f7a4dd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.644420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-820d64e4-2d4b-4b34-a41e-9453f7a4dd5e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:35.645640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-820d64e4-2d4b-4b34-a41e-9453f7a4dd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:35.724751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1046/2109] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:35.736155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:35.737915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:35.806433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41984afa-4ccd-4015-9f75-6037df297f14 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:35.811707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:35.812340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:35.812619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:35.812890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:35.824158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:35.826026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:35.842939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1064/2110] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:36.499228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21582a3-afa9-44ce-8ab5-348cf0882df7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:36.500182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21582a3-afa9-44ce-8ab5-348cf0882df7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:36.500616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21582a3-afa9-44ce-8ab5-348cf0882df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:36.530916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c21582a3-afa9-44ce-8ab5-348cf0882df7 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:36.532655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c21582a3-afa9-44ce-8ab5-348cf0882df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:36.570056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:36.651553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1047/2111] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:09:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:36.719508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:36.721134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:36.850121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911db42-7854-49fe-a298-ece4d041e317 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:36.862873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:36.863829 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:36.864275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:36.864701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:36.883900 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:36.885857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:36.907667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1065/2112] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:09:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:09:37.755227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8e5c6-6bba-4870-a5a0-1d189177514b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:37.756128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8e5c6-6bba-4870-a5a0-1d189177514b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:37.756515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8e5c6-6bba-4870-a5a0-1d189177514b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:37.786265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-80a8e5c6-6bba-4870-a5a0-1d189177514b None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:37.787652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-80a8e5c6-6bba-4870-a5a0-1d189177514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:37.797411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1048/2113] 10.222.0.94 () {58 vars in 967 bytes} [Mon Aug 29 19:09:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:37.812888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:37.824340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e14bd-391f-4ea0-b5ba-6f738e0005c6 service swift] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:37.835407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:37.835407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:37.835407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:37.835407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:37.846070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:37.847879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:37.858195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1066/2114] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:09:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:09:38.128204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9dbd7-dbde-4fe9-ac5d-47c99bc4e10a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:38.128499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9dbd7-dbde-4fe9-ac5d-47c99bc4e10a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:38.128871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9dbd7-dbde-4fe9-ac5d-47c99bc4e10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:38.152445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b7e9dbd7-dbde-4fe9-ac5d-47c99bc4e10a None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:38.153913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b7e9dbd7-dbde-4fe9-ac5d-47c99bc4e10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:38.168107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1049/2115] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:09:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:38.182421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:38.207846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f5072-cb0a-4f80-aacc-003c4e61882a service swift] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:38.214186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:38.214815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:38.215143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:38.215428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:38.225607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:38.227826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:38.241337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1067/2116] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:09:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:09:38.775138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd2d1e-bc4e-40c1-bcd4-f76fe788826e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:38.776139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd2d1e-bc4e-40c1-bcd4-f76fe788826e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:38.776570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd2d1e-bc4e-40c1-bcd4-f76fe788826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:38.802683 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7ccd2d1e-bc4e-40c1-bcd4-f76fe788826e None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:38.805489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7ccd2d1e-bc4e-40c1-bcd4-f76fe788826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:38.827386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1050/2117] 10.222.0.94 () {58 vars in 967 bytes} [Mon Aug 29 19:09:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:38.842186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:38.861159 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf316e94-784d-4810-8e30-a479f585770e service swift] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:38.866921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:38.867793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:38.868180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:38.868560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:38.879978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:38.882055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:38.903431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1068/2118] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:09:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:09:41.225734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06a749-8adb-4b68-9697-6ce3e25211d3 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:41.226183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06a749-8adb-4b68-9697-6ce3e25211d3 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:41.226296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06a749-8adb-4b68-9697-6ce3e25211d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:41.264465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ce06a749-8adb-4b68-9697-6ce3e25211d3 None None] MFA Rules not processed for user `8f1c0b2189a041fdb8dbe2a0290ffac8`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:41.265764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ce06a749-8adb-4b68-9697-6ce3e25211d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:41.340317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1051/2119] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:41.344957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:41.356936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e10db-7b73-4228-be56-3a446a838a25 service swift] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:41.361822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:41.362438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:41.362823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:41.363256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:41.363995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:41.366012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:41.377769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:41.399623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1069/2120] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:41 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:41.403259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06a749-8adb-4b68-9697-6ce3e25211d3 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:41.416499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06a749-8adb-4b68-9697-6ce3e25211d3 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:41.506851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:41.507754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:41.508003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:41.508270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:41.509022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:41.510949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:41.534579 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1052/2121] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:41 2022] POST /identity/v3/users => generated 295 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:41.538792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:41.550522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34c9f6-a082-4ba4-9d7c-ce1e0b602aa7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:41.555823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:41.556525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:41.556808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:41.557074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:41.557730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:41.559365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:41.570444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1070/2122] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:41.574723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:41.585356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1fda7-88f3-49d0-98c2-94d682f63f14 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:41.592317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:41.593252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:41.593530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:41.593803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] PATH_INFO: `/v3/projects/35c86e18d47449a6bfb9a93397cec4e5/users/f181016e9d094ba98cf0cceac6ad5ff2/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:41.611187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c86e18d47449a6bfb9a93397cec4e5, user_id=f181016e9d094ba98cf0cceac6ad5ff2, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:41.613928 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:41.629243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1053/2123] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:41 2022] PUT /identity/v3/projects/35c86e18d47449a6bfb9a93397cec4e5/users/f181016e9d094ba98cf0cceac6ad5ff2/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:41.633640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:41.648176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbc64c-1981-4031-a687-b98f02fb643b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:41.712748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:41.712748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:41.713152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:41.713152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:41.714473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:41.716390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:41.727788 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1071/2124] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:41.731816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:41.743519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5c0f5-6e1f-4ae9-8cd8-e0490ee7e235 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:41.748602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58e030-2497-4a51-ad5f-2be35ed7ab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:41.749279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58e030-2497-4a51-ad5f-2be35ed7ab8d admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:41.749560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58e030-2497-4a51-ad5f-2be35ed7ab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:41.749834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58e030-2497-4a51-ad5f-2be35ed7ab8d admin admin] PATH_INFO: `/v3/projects/35c86e18d47449a6bfb9a93397cec4e5/users/f181016e9d094ba98cf0cceac6ad5ff2/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:41.751856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58e030-2497-4a51-ad5f-2be35ed7ab8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c86e18d47449a6bfb9a93397cec4e5, user_id=f181016e9d094ba98cf0cceac6ad5ff2, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:41.754538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58e030-2497-4a51-ad5f-2be35ed7ab8d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:41.758859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:41.768586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1054/2125] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:41 2022] PUT /identity/v3/projects/35c86e18d47449a6bfb9a93397cec4e5/users/f181016e9d094ba98cf0cceac6ad5ff2/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:43.562637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:43.564637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:43.595867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63902c3-75c6-4635-bfe4-31c7ab303bf1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:43.691465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:43.692522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:43.693017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:43.693505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] PATH_INFO: `/v3/users/4c27047dbc2c40de89017865e41a0dc6` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:43.694860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c27047dbc2c40de89017865e41a0dc6)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:43.698047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:43.778728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c27047dbc2c40de89017865e41a0dc6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:43.793931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c27047dbc2c40de89017865e41a0dc6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:43.795030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c27047dbc2c40de89017865e41a0dc6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:43.821684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c27047dbc2c40de89017865e41a0dc6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:43.873917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:43.893876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c27047dbc2c40de89017865e41a0dc6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:43.904775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1072/2126] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:43 2022] DELETE /identity/v3/users/4c27047dbc2c40de89017865e41a0dc6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:43.910343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:43.910868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:43.911145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:43.967056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:43.968680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:44.101081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1055/2127] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:09:44.112591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:44.114668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:44.236814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd37806-3f24-4a0c-857a-54946e66a4db admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:44.253185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39852c-e580-44f3-95af-a7497b4eb73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:44.253185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39852c-e580-44f3-95af-a7497b4eb73b service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:44.254717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39852c-e580-44f3-95af-a7497b4eb73b service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:44.259093 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39852c-e580-44f3-95af-a7497b4eb73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:44.277243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39852c-e580-44f3-95af-a7497b4eb73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:44.279276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39852c-e580-44f3-95af-a7497b4eb73b service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:44.292811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1073/2128] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:44.582705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:44.584184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:44.603625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d2cc3-caa2-43cf-bcc4-a3f53d1cec24 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:44.614913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:44.616022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:44.616463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:44.616988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] PATH_INFO: `/v3/projects/0f594096296743c583bf3f8e0c1f5110` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:44.619769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f594096296743c583bf3f8e0c1f5110)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:44.622539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:44.741915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f594096296743c583bf3f8e0c1f5110', 'request_id': 'req-643f13f6-0e30-49d7-814f-c7597feea8b0'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:44.766472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f594096296743c583bf3f8e0c1f5110', 'request_id': 'req-643f13f6-0e30-49d7-814f-c7597feea8b0'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:44.788422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f594096296743c583bf3f8e0c1f5110', 'request_id': 'req-643f13f6-0e30-49d7-814f-c7597feea8b0'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:44.820159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] The token cache is being invalidate because project 0f594096296743c583bf3f8e0c1f5110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:44.822584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:44.823880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-643f13f6-0e30-49d7-814f-c7597feea8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:44.842107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1056/2129] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:44 2022] DELETE /identity/v3/projects/0f594096296743c583bf3f8e0c1f5110 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:44.847062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92d75c-913c-4bb7-8601-7c34e86eeeda None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:44.847485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92d75c-913c-4bb7-8601-7c34e86eeeda None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:44.847718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92d75c-913c-4bb7-8601-7c34e86eeeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:44.894028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ab92d75c-913c-4bb7-8601-7c34e86eeeda None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:44.895692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ab92d75c-913c-4bb7-8601-7c34e86eeeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:45.102810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1074/2130] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:45.112375 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4135df7-3038-490e-b8b0-874c62438b20 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.112857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4135df7-3038-490e-b8b0-874c62438b20 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.113139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4135df7-3038-490e-b8b0-874c62438b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.174755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c4135df7-3038-490e-b8b0-874c62438b20 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:45.177805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c4135df7-3038-490e-b8b0-874c62438b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:45.203544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1057/2131] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:45.205102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92d75c-913c-4bb7-8601-7c34e86eeeda None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:45.220969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92d75c-913c-4bb7-8601-7c34e86eeeda None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:45.236351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:45.237034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.237312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.238125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.239106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:45.239948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:45.243491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:45.268168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1075/2132] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:09:45.275469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4135df7-3038-490e-b8b0-874c62438b20 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:45.312076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4135df7-3038-490e-b8b0-874c62438b20 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:45.328587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:45.330742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.331111 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.333004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.334201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:45.340417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:45.400249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1058/2133] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:45 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:45.405199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:45.420758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4c879-edf5-428e-a6c0-a982e3299a15 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:45.553379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:45.554153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.554515 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.555634 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.556627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:45.558854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:45.595519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1076/2134] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:45 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:45.599819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:45.619857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fbf3a-1518-4d63-84f9-55aa4b3f7f72 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:45.627874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:45.629269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.629850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.630271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.631223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:45.633270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:45.646950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1059/2135] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:45.651288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:45.666186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e2eea-d83b-4965-8e6a-0f9bfb608021 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:45.672404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6113c5-72d3-4d84-b81b-63e105c79b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:45.673265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6113c5-72d3-4d84-b81b-63e105c79b3e admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.673877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6113c5-72d3-4d84-b81b-63e105c79b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.674295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6113c5-72d3-4d84-b81b-63e105c79b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.675366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6113c5-72d3-4d84-b81b-63e105c79b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:45.677589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6113c5-72d3-4d84-b81b-63e105c79b3e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:45.711987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1077/2136] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:45.715720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:45.734153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42710218-d88c-4f5a-a23f-5c05141633b5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:45.749715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:45.750716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.752072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.753856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] PATH_INFO: `/v3/projects/f0f4bcf841c942a9928f5b343b5fb8bb/users/82fb5d585b5d460bb258f929057a4093/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.794950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f4bcf841c942a9928f5b343b5fb8bb, user_id=82fb5d585b5d460bb258f929057a4093, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:45.798330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:45.826270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1060/2137] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:45 2022] PUT /identity/v3/projects/f0f4bcf841c942a9928f5b343b5fb8bb/users/82fb5d585b5d460bb258f929057a4093/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:45.831718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:45.832426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:45.832695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:45.843673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:45.865492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:45.868189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:46.042169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1078/2138] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:46.070747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:46.073089 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:46.114894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:46.206121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a265d3-fdc8-4119-8448-604fa60cc2c6 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:46.209989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44a973-a57f-4a90-a48f-5a1428986b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:46.210553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44a973-a57f-4a90-a48f-5a1428986b19 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:46.210836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44a973-a57f-4a90-a48f-5a1428986b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:46.211817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44a973-a57f-4a90-a48f-5a1428986b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:46.241036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44a973-a57f-4a90-a48f-5a1428986b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:46.245280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44a973-a57f-4a90-a48f-5a1428986b19 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:46.285771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1061/2139] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:47.520437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:47.522805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:47.550283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49da295-8de3-462a-85d7-e7f208388d2e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:47.566405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:47.567192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:47.567547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:47.567894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] PATH_INFO: `/v3/users/e9820162f8d74e1ab1865d80d5dc2f5d` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:47.569311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9820162f8d74e1ab1865d80d5dc2f5d)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:47.576817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:47.703950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9820162f8d74e1ab1865d80d5dc2f5d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:47.731153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9820162f8d74e1ab1865d80d5dc2f5d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:47.732168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9820162f8d74e1ab1865d80d5dc2f5d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:47.766803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9820162f8d74e1ab1865d80d5dc2f5d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:47.783824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9820162f8d74e1ab1865d80d5dc2f5d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:47.796353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1079/2140] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:47 2022] DELETE /identity/v3/users/e9820162f8d74e1ab1865d80d5dc2f5d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:47.811262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04bd79-0f73-41f1-9b72-ce493a3414b8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:47.817165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04bd79-0f73-41f1-9b72-ce493a3414b8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:47.817671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04bd79-0f73-41f1-9b72-ce493a3414b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:47.874087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-fb04bd79-0f73-41f1-9b72-ce493a3414b8 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:47.876910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fb04bd79-0f73-41f1-9b72-ce493a3414b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:48.047808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1062/2141] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:48.079282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:48.081203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:48.234924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75a492-defd-4b50-b5f0-4ea41c29dcc1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:48.240835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:48.241804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:48.245427 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:48.245924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:48.284552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:48.286214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:48.338682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1080/2142] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:49.541975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:49.542858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:49.543376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:49.613351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:49.615349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:49.635355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] MFA Rules not processed for user `f181016e9d094ba98cf0cceac6ad5ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:49.637695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:49.638698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6bae-27b0-4713-90ae-7a4fa564d1fb service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:49.651812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:49.652919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:49.653361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:49.655958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] PATH_INFO: `/v3/projects/f9c3a16a2dd24da0a7d5adde8c284fbf` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:49.658744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c3a16a2dd24da0a7d5adde8c284fbf)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:49.661086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:49.693154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:49.812931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c3a16a2dd24da0a7d5adde8c284fbf', 'request_id': 'req-86768593-c1d8-4b61-99c5-0e07ca06227f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:49.842094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c3a16a2dd24da0a7d5adde8c284fbf', 'request_id': 'req-86768593-c1d8-4b61-99c5-0e07ca06227f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:49.856591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c3a16a2dd24da0a7d5adde8c284fbf', 'request_id': 'req-86768593-c1d8-4b61-99c5-0e07ca06227f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:49.889341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] The token cache is being invalidate because project f9c3a16a2dd24da0a7d5adde8c284fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:49.889637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:49.890305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:49.892058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1081/2143] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:49 2022] DELETE /identity/v3/projects/f9c3a16a2dd24da0a7d5adde8c284fbf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:49.896222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:49.901310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:49.904037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1063/2144] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:49.919163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:49.920920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:50.078100 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:50.112998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86768593-c1d8-4b61-99c5-0e07ca06227f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:50.120946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:50.121581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:50.121820 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:50.131125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] PATH_INFO: `/v3/users/7f4ad20324134965b110943230be35e0` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:50.133762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4ad20324134965b110943230be35e0)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:50.136054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:50.151984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fea0d4-dedd-4d58-8e26-289641536b3d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:50.309885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4ad20324134965b110943230be35e0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:50.336143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4ad20324134965b110943230be35e0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:50.337719 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4ad20324134965b110943230be35e0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:50.343592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dce599-f722-4143-97bf-32eabb1484f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:50.344228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dce599-f722-4143-97bf-32eabb1484f5 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:50.344511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dce599-f722-4143-97bf-32eabb1484f5 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:50.344785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dce599-f722-4143-97bf-32eabb1484f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:50.355869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c8dce599-f722-4143-97bf-32eabb1484f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:50.385348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4ad20324134965b110943230be35e0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:50.396579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4ad20324134965b110943230be35e0'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:50.404615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1082/2145] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:49 2022] DELETE /identity/v3/users/7f4ad20324134965b110943230be35e0 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:50.499092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dce599-f722-4143-97bf-32eabb1484f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:50.500900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dce599-f722-4143-97bf-32eabb1484f5 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:50.522399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1064/2146] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:09:51.189284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:51.191798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:51.253886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:51.363509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4cda9-de40-4f55-b58e-83999476cd0b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:51.375562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:51.376595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:51.376595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:51.382672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] PATH_INFO: `/v3/projects/7eb85ae214e647f690b1b8493a83ee35` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:51.386517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb85ae214e647f690b1b8493a83ee35)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:51.394992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:51.522053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb85ae214e647f690b1b8493a83ee35', 'request_id': 'req-420325e8-c56d-4aee-ba8b-df8c7de27692'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:51.536017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb85ae214e647f690b1b8493a83ee35', 'request_id': 'req-420325e8-c56d-4aee-ba8b-df8c7de27692'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:51.552980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb85ae214e647f690b1b8493a83ee35', 'request_id': 'req-420325e8-c56d-4aee-ba8b-df8c7de27692'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:51.578363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] The token cache is being invalidate because project 7eb85ae214e647f690b1b8493a83ee35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:51.578672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:51.579337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-420325e8-c56d-4aee-ba8b-df8c7de27692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:51.581096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1083/2147] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:51 2022] DELETE /identity/v3/projects/7eb85ae214e647f690b1b8493a83ee35 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:51.587691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690706-628a-40f0-b421-5512a63681c7 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:51.588337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690706-628a-40f0-b421-5512a63681c7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:51.588337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690706-628a-40f0-b421-5512a63681c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:51.632569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ae690706-628a-40f0-b421-5512a63681c7 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:51.634585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ae690706-628a-40f0-b421-5512a63681c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:51.780987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1065/2148] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:51.790615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c19c-df2c-42c1-a464-52356040beed None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:51.790989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c19c-df2c-42c1-a464-52356040beed None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:51.791336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c19c-df2c-42c1-a464-52356040beed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:51.825374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-5fe9c19c-df2c-42c1-a464-52356040beed None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:51.827775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-5fe9c19c-df2c-42c1-a464-52356040beed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:51.857898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1084/2149] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:51.862170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690706-628a-40f0-b421-5512a63681c7 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:51.874897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690706-628a-40f0-b421-5512a63681c7 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:51.896638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:51.897299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:51.897557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:51.897831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:51.898670 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:51.899528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:51.901304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:51.951600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1066/2150] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:09:51.957301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c19c-df2c-42c1-a464-52356040beed None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:51.985135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c19c-df2c-42c1-a464-52356040beed None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:51.991538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:51.992368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:51.992741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:51.993574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:51.995108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:51.996735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:52.052777 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1085/2151] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:51 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:52.059503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:52.100796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcb1c0-0cf5-4f6c-800c-664db666df24 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:52.108831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:52.328910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:52.330513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:52.330776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:52.331238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:52.333192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:52.334188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:52.401577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1067/2152] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:52 2022] POST /identity/v3/users => generated 293 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:52.404527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:52.422892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7a45a-17f5-4649-aab9-1acf14c35759 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:52.428107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:52.429063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:52.429882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:52.430474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:52.431599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:52.433505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:52.459808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1086/2153] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:52.463998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:52.484587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70038e-99c1-4151-9026-fd55b6f18eb0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:52.502347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9b06f-a6f6-42ee-b0dc-bd650bcfee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:52.503317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b06f-a6f6-42ee-b0dc-bd650bcfee01 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:52.503603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b06f-a6f6-42ee-b0dc-bd650bcfee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:52.503938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b06f-a6f6-42ee-b0dc-bd650bcfee01 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:52.504604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9b06f-a6f6-42ee-b0dc-bd650bcfee01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:52.512400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9b06f-a6f6-42ee-b0dc-bd650bcfee01 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:52.552785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1068/2154] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:52.556436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:52.583822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bfd3-4545-41b4-80ef-d631045c071b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:52.614928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:52.621039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:52.625035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:52.625035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] PATH_INFO: `/v3/projects/24f844238a6a4e1eb2208c86afcdd8ea/users/20a7481ed74f4ede89f229dd133cde72/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:52.665770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f844238a6a4e1eb2208c86afcdd8ea, user_id=20a7481ed74f4ede89f229dd133cde72, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:52.668908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:52.703321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1087/2155] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:52 2022] PUT /identity/v3/projects/24f844238a6a4e1eb2208c86afcdd8ea/users/20a7481ed74f4ede89f229dd133cde72/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:52.705045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cc718-6e26-4096-9603-8f6545ab262a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:52.705503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cc718-6e26-4096-9603-8f6545ab262a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:52.705736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cc718-6e26-4096-9603-8f6545ab262a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:52.805581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-215cc718-6e26-4096-9603-8f6545ab262a None None] MFA Rules not processed for user `20a7481ed74f4ede89f229dd133cde72`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:52.808166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-215cc718-6e26-4096-9603-8f6545ab262a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:52.977348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1069/2156] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 29 19:09:52.978764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:53.093129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:53.131046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06718a-fb05-43b7-9439-d0bd90e0225d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:53.136924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:53.137639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:53.146331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:53.147167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:53.147675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:53.151354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:53.228973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1088/2157] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:52 2022] POST /identity/v3/projects => generated 387 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:53.232648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cc718-6e26-4096-9603-8f6545ab262a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:53.256605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cc718-6e26-4096-9603-8f6545ab262a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:53.507223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:53.507903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:53.508852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:53.509064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:53.509695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:53.512918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:53.605640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1070/2158] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:53 2022] POST /identity/v3/users => generated 300 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:53.606665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:53.621975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9997f-bffc-44b7-b2ae-ce3a0105a7c1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:53.636652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:53.637307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:53.637767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:53.638224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:53.639047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:53.650619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:53.682504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:53.683449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1089/2159] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:53.697280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:53.716325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff487e6b-2308-4a72-bbec-96545a5b4c30 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:53.735903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:53.737036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:53.738134 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:53.740229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] PATH_INFO: `/v3/projects/bf4c15811238428797929a4b00bbcdd8/users/4927065c5d8a49c3a2ef82d8de8808df/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:53.791261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4c15811238428797929a4b00bbcdd8, user_id=4927065c5d8a49c3a2ef82d8de8808df, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:53.810712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:53.851191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1071/2160] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:53 2022] PUT /identity/v3/projects/bf4c15811238428797929a4b00bbcdd8/users/4927065c5d8a49c3a2ef82d8de8808df/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:53.853032 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:53.866722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656069e0-59c0-4803-9c30-0c94fe52c2ed admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:54.022105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bae6ee-cd33-47c7-98a1-c70d8ca9f3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:54.022747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bae6ee-cd33-47c7-98a1-c70d8ca9f3fa admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:54.023013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bae6ee-cd33-47c7-98a1-c70d8ca9f3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:54.023324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bae6ee-cd33-47c7-98a1-c70d8ca9f3fa admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:54.024409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bae6ee-cd33-47c7-98a1-c70d8ca9f3fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:54.026235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bae6ee-cd33-47c7-98a1-c70d8ca9f3fa admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:54.049497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1090/2161] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:54.050214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:54.068707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f07fd-3d68-464b-a2d5-81ce6c2aee30 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:54.083278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:54.084527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:54.084527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:54.084527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] PATH_INFO: `/v3/projects/bf4c15811238428797929a4b00bbcdd8/users/4927065c5d8a49c3a2ef82d8de8808df/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:54.094083 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4c15811238428797929a4b00bbcdd8, user_id=4927065c5d8a49c3a2ef82d8de8808df, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:54.097842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:54.156140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1072/2162] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:54 2022] PUT /identity/v3/projects/bf4c15811238428797929a4b00bbcdd8/users/4927065c5d8a49c3a2ef82d8de8808df/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:54.172464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4770d9-22cf-4734-9b73-436e6abf8b4f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:54.172464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4770d9-22cf-4734-9b73-436e6abf8b4f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:54.172854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4770d9-22cf-4734-9b73-436e6abf8b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:54.278772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ca4770d9-22cf-4734-9b73-436e6abf8b4f None None] MFA Rules not processed for user `4927065c5d8a49c3a2ef82d8de8808df`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:54.280217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ca4770d9-22cf-4734-9b73-436e6abf8b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:54.419375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1091/2163] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:54.433630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:54.436025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:54.522720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e9255-639f-4d45-ac14-9546ae3644d0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:54.528873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe27b86-4337-4cbb-b2fe-f52316e0d278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:54.529515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe27b86-4337-4cbb-b2fe-f52316e0d278 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:54.529793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe27b86-4337-4cbb-b2fe-f52316e0d278 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:54.530063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe27b86-4337-4cbb-b2fe-f52316e0d278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:54.556738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe27b86-4337-4cbb-b2fe-f52316e0d278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:54.558572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe27b86-4337-4cbb-b2fe-f52316e0d278 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:54.588395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1073/2164] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:54.704547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4770d9-22cf-4734-9b73-436e6abf8b4f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:54.777821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:54.808493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4770d9-22cf-4734-9b73-436e6abf8b4f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:54.826545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:54.827568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:54.828551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:54.828551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] PATH_INFO: `/v3/users/20a7481ed74f4ede89f229dd133cde72` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:54.829983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a7481ed74f4ede89f229dd133cde72)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:54.832762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:55.003441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a7481ed74f4ede89f229dd133cde72'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:55.014897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a7481ed74f4ede89f229dd133cde72'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:55.018890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a7481ed74f4ede89f229dd133cde72'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:55.051390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a7481ed74f4ede89f229dd133cde72'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:55.067651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a7481ed74f4ede89f229dd133cde72'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:55.079973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1092/2165] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:54 2022] DELETE /identity/v3/users/20a7481ed74f4ede89f229dd133cde72 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:55.084836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97ab8c-7cf7-400b-8cbf-af67f165bdac None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:55.087952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97ab8c-7cf7-400b-8cbf-af67f165bdac None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:55.088389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97ab8c-7cf7-400b-8cbf-af67f165bdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:55.136660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cc97ab8c-7cf7-400b-8cbf-af67f165bdac None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:55.138869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cc97ab8c-7cf7-400b-8cbf-af67f165bdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:55.257561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1074/2166] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:55.268950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:55.272105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:55.410090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84421c-2ace-41cd-8139-20e5fc6e0036 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:55.417085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:55.417834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:55.417985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:55.418257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:55.452043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:55.453441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:55.495467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1093/2167] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:09:56.190145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:56.190760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:56.191204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:56.256499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:56.258750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:56.293707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc30d2e-8af1-4c9c-910c-3486073935fd service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:56.303536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:56.304460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:56.305716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:56.306212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] PATH_INFO: `/v3/projects/24f844238a6a4e1eb2208c86afcdd8ea` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:56.307691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] MFA Rules not processed for user `82fb5d585b5d460bb258f929057a4093`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:56.308702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f844238a6a4e1eb2208c86afcdd8ea)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:56.308930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:56.312127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:56.417518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f844238a6a4e1eb2208c86afcdd8ea', 'request_id': 'req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.425449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f844238a6a4e1eb2208c86afcdd8ea', 'request_id': 'req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.434081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:56.438333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f844238a6a4e1eb2208c86afcdd8ea', 'request_id': 'req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.463754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:56.467902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] The token cache is being invalidate because project 24f844238a6a4e1eb2208c86afcdd8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:56.468323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.469762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.472654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1075/2168] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:56.474064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:56.476416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:56.482320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1094/2169] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:56 2022] DELETE /identity/v3/projects/24f844238a6a4e1eb2208c86afcdd8ea => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:56.487027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:56.490677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:56.517733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3f12e-3f82-4fe5-8b2f-1f6c9821d55c None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:56.603806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cdc71-bb6e-49ee-adb5-c71edf4712d4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:56.610731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:56.611668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:56.611961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:56.612662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:56.613641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:56.614523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:56.615273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:56.615741 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:56.615878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:56.616137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] PATH_INFO: `/v3/users/4927065c5d8a49c3a2ef82d8de8808df` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:56.620085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4927065c5d8a49c3a2ef82d8de8808df)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:56.622706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:56.662868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1076/2170] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:56 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:56.668366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:56.670829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:56.700033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4927065c5d8a49c3a2ef82d8de8808df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.715162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4927065c5d8a49c3a2ef82d8de8808df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.716445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4927065c5d8a49c3a2ef82d8de8808df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.746049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4927065c5d8a49c3a2ef82d8de8808df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.757571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4927065c5d8a49c3a2ef82d8de8808df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:56.775138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca192f-25a0-42a0-b3ee-9a9e35e86692 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:56.776673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1095/2171] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:09:56 2022] DELETE /identity/v3/users/4927065c5d8a49c3a2ef82d8de8808df => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:56.893568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:56.894721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:56.895787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:56.896188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:56.897031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:56.899880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:56.939637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1077/2172] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:56 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:56.944022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:56.945964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:56.982604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9d62-c440-4728-aa1c-23e828f906f8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:56.984765 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:56.985393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:56.985671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:56.985939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:56.988108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:56.989838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:57.008767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1096/2173] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:09:57.013970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:57.027698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34114-2bcc-420b-bfce-276682d23c5c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:57.034075 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:57.035193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:57.035527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:57.035821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] PATH_INFO: `/v3/projects/edf46f0b532246c5ad132be112175ca9/users/aeac5df40d57465e820d3b961633c5ac/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:57.059034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf46f0b532246c5ad132be112175ca9, user_id=aeac5df40d57465e820d3b961633c5ac, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:57.061895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:57.091972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:57.116160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1078/2174] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:57 2022] PUT /identity/v3/projects/edf46f0b532246c5ad132be112175ca9/users/aeac5df40d57465e820d3b961633c5ac/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:09:57.116552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:57.133957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c815b-3b4c-4d16-a30f-f6f1114a0f9c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:57.236703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:57.239434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:57.255877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:57.275349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:57.276290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:57.279364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:57.280544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:57.282330 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:57.284592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:57.300879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1097/2175] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:57.302077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:57.314115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a47b48-0be2-42ba-9b10-2a66e4db2362 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:57.318653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608fde4-e329-4933-bc9f-5492a6315dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:57.319488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608fde4-e329-4933-bc9f-5492a6315dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:57.319767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608fde4-e329-4933-bc9f-5492a6315dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:57.320038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608fde4-e329-4933-bc9f-5492a6315dc3 admin admin] PATH_INFO: `/v3/projects/edf46f0b532246c5ad132be112175ca9/users/aeac5df40d57465e820d3b961633c5ac/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:57.323192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608fde4-e329-4933-bc9f-5492a6315dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf46f0b532246c5ad132be112175ca9, user_id=aeac5df40d57465e820d3b961633c5ac, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:57.325978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608fde4-e329-4933-bc9f-5492a6315dc3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:57.342264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1098/2176] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:57 2022] PUT /identity/v3/projects/edf46f0b532246c5ad132be112175ca9/users/aeac5df40d57465e820d3b961633c5ac/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:57.393886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d91862-eff7-4b97-86de-89ae879821ae admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:57.551733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:57.552477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:57.552786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:57.553281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] PATH_INFO: `/v3/projects/bf4c15811238428797929a4b00bbcdd8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:57.554263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf4c15811238428797929a4b00bbcdd8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:57.558930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:57.675550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf4c15811238428797929a4b00bbcdd8', 'request_id': 'req-c55cbd60-67fa-44ee-b435-d85cd05d369e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:57.694423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf4c15811238428797929a4b00bbcdd8', 'request_id': 'req-c55cbd60-67fa-44ee-b435-d85cd05d369e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:57.716495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf4c15811238428797929a4b00bbcdd8', 'request_id': 'req-c55cbd60-67fa-44ee-b435-d85cd05d369e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:57.737772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] The token cache is being invalidate because project bf4c15811238428797929a4b00bbcdd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:09:57.738088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:57.741623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c55cbd60-67fa-44ee-b435-d85cd05d369e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:09:57.747827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1079/2177] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:09:57 2022] DELETE /identity/v3/projects/bf4c15811238428797929a4b00bbcdd8 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:57.752859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e2ee-6f86-4c66-83ba-3c06e4e600d5 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:57.753175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e2ee-6f86-4c66-83ba-3c06e4e600d5 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:57.753576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e2ee-6f86-4c66-83ba-3c06e4e600d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:57.817847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-3154e2ee-6f86-4c66-83ba-3c06e4e600d5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:57.820423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3154e2ee-6f86-4c66-83ba-3c06e4e600d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:58.017048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1099/2178] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:58.025827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925a044-869b-4c11-ab3b-9b1c173579b1 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.026152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925a044-869b-4c11-ab3b-9b1c173579b1 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.026500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925a044-869b-4c11-ab3b-9b1c173579b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.090744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-4925a044-869b-4c11-ab3b-9b1c173579b1 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:58.092995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4925a044-869b-4c11-ab3b-9b1c173579b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:58.115272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1080/2179] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:58.121492 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e2ee-6f86-4c66-83ba-3c06e4e600d5 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:58.132299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e2ee-6f86-4c66-83ba-3c06e4e600d5 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:58.137815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:58.138722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.139129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.139507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.141561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:09:58.141981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:58.145423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:58.184699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1100/2180] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:09:58.189180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925a044-869b-4c11-ab3b-9b1c173579b1 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:58.205331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925a044-869b-4c11-ab3b-9b1c173579b1 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:58.218503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:58.219809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.220322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.220694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.221447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:58.223826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:58.308042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1081/2181] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:58 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:58.311999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:58.328051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed052943-aff2-45bd-a50d-c31aaa1001b8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:58.448166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:58.448948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.454504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.455026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.455841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:58.461509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:58.511257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1101/2182] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:58 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:58.511812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:58.527169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae6acf-d2ad-430f-87a5-47903cb9957f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:58.546794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:58.547483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.547767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.549504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.549504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:58.551893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:58.556679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:58.579419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1082/2183] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:58.582493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:58.603760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244766e-dac3-483b-9c12-12fcdf5880c9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:58.614064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c6b19-900f-4a35-b270-0c5944e46611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:58.614637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c6b19-900f-4a35-b270-0c5944e46611 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.614986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c6b19-900f-4a35-b270-0c5944e46611 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.615375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c6b19-900f-4a35-b270-0c5944e46611 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.616557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c6b19-900f-4a35-b270-0c5944e46611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:58.619040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c6b19-900f-4a35-b270-0c5944e46611 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:58.643712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1102/2184] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:58.648029 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:58.670680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cc282-b15e-4a5e-9a28-6a014e9c901f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:58.678114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:58.678879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.679193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.679549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] PATH_INFO: `/v3/projects/7a1a864c665b495a910a93db3ed93b87/users/328de220e50447a8809819a3d9b313ac/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.706628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1a864c665b495a910a93db3ed93b87, user_id=328de220e50447a8809819a3d9b313ac, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:58.710416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:58.746900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1083/2185] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:58 2022] PUT /identity/v3/projects/7a1a864c665b495a910a93db3ed93b87/users/328de220e50447a8809819a3d9b313ac/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:58.749089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d683f-5d63-4066-a5d4-2658f4463372 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:58.749269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d683f-5d63-4066-a5d4-2658f4463372 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:58.749538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d683f-5d63-4066-a5d4-2658f4463372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:58.789390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-db8d683f-5d63-4066-a5d4-2658f4463372 None None] MFA Rules not processed for user `328de220e50447a8809819a3d9b313ac`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:58.790712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-db8d683f-5d63-4066-a5d4-2658f4463372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:58.871650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:09:58.923355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1103/2186] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:58.924128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:59.022776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e3cee-0aa6-4b14-a74b-d18a6825c5ce admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:59.037495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:59.038240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:59.039348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:59.040239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:59.041140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:59.043629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:59.087485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1084/2187] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:58 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:59.091807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d683f-5d63-4066-a5d4-2658f4463372 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:59.104637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d683f-5d63-4066-a5d4-2658f4463372 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:59.236444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:59.237078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:59.237361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:59.237636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:59.238290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:59.240373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:59.270013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1104/2188] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:59 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:09:59.274169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:59.291394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638641-f22f-4cd7-880d-f3cff10e6da3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:59.299991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:59.301399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:59.309882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:59.310403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:59.311724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:59.314121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:59.340861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1085/2189] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:59.345072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:59.363134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b87982-8b51-456b-8eb9-b0436adfdc38 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:59.380151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4af29c-124e-4fe1-8dd6-ce52ccbdcde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:59.380839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4af29c-124e-4fe1-8dd6-ce52ccbdcde4 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:59.381530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4af29c-124e-4fe1-8dd6-ce52ccbdcde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:59.381872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4af29c-124e-4fe1-8dd6-ce52ccbdcde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:59.384424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4af29c-124e-4fe1-8dd6-ce52ccbdcde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:59.388179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4af29c-124e-4fe1-8dd6-ce52ccbdcde4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:59.413436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1105/2190] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:09:59.418055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:59.438549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a44b53-5eaa-49cb-9dfc-8bc3ebf25d65 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:59.453999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:59.454710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:59.455206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:59.455491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] PATH_INFO: `/v3/projects/15e7858f3c154ec8a38c0069f83853a6/users/2d8017147b224017a837026cbab41135/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:59.496312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e7858f3c154ec8a38c0069f83853a6, user_id=2d8017147b224017a837026cbab41135, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:59.499038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:59.550478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1086/2191] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:09:59 2022] PUT /identity/v3/projects/15e7858f3c154ec8a38c0069f83853a6/users/2d8017147b224017a837026cbab41135/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:09:59.550977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9ff8-c0bb-4a4d-8f30-12b50d611d70 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:59.551331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9ff8-c0bb-4a4d-8f30-12b50d611d70 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:59.551616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9ff8-c0bb-4a4d-8f30-12b50d611d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:59.616575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ceae9ff8-c0bb-4a4d-8f30-12b50d611d70 None None] MFA Rules not processed for user `2d8017147b224017a837026cbab41135`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:09:59.618633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ceae9ff8-c0bb-4a4d-8f30-12b50d611d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:09:59.754534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1106/2192] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:09:59.756830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:59.870273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a4ad-8815-430c-b669-ccde195bea8b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:09:59.874970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:09:59.875574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:09:59.875795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:09:59.876072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:09:59.876851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:09:59.879350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:09:59.941710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1087/2193] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:09:59 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:09:59.944017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9ff8-c0bb-4a4d-8f30-12b50d611d70 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:09:59.996879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9ff8-c0bb-4a4d-8f30-12b50d611d70 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:00.162948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:00.221615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:00.222679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:00.223165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:00.223590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:00.224549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:00.230463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:00.286378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1107/2194] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:09:59 2022] POST /identity/v3/users => generated 300 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:00.287189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:00.300160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e6756-0b8c-4167-b4ad-8f87227e6e59 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:00.331180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:00.336277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:00.337261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:00.337727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:00.338422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:00.340106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:00.342404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:00.372745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1088/2195] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:10:00.377140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:00.398773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0ffc-f535-44e2-aa52-f0ceae82e461 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:00.424909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:00.426968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:00.428282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:00.428639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] PATH_INFO: `/v3/projects/3568bcaeec69428ba4cb018cc8dd6a00/users/e7cdebe112994f999c69b336292163ed/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:00.461779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3568bcaeec69428ba4cb018cc8dd6a00, user_id=e7cdebe112994f999c69b336292163ed, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:00.466428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:00.495154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1108/2196] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:00 2022] PUT /identity/v3/projects/3568bcaeec69428ba4cb018cc8dd6a00/users/e7cdebe112994f999c69b336292163ed/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:00.498838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:00.533395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07611a7a-1a7d-4a5c-8a82-8d82358d2a96 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:00.644785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee1fbe-5174-4dd3-b3af-00fff371c0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:00.645684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee1fbe-5174-4dd3-b3af-00fff371c0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:00.646470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee1fbe-5174-4dd3-b3af-00fff371c0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:00.646907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee1fbe-5174-4dd3-b3af-00fff371c0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:00.647741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee1fbe-5174-4dd3-b3af-00fff371c0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:00.649709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee1fbe-5174-4dd3-b3af-00fff371c0e0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:00.685147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1089/2197] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:00.689303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:00.706324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0f15b-b528-4e32-a37f-2d26b89a790d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:00.723961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:00.724647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:00.724927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:00.725194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] PATH_INFO: `/v3/projects/3568bcaeec69428ba4cb018cc8dd6a00/users/e7cdebe112994f999c69b336292163ed/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:00.728248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3568bcaeec69428ba4cb018cc8dd6a00, user_id=e7cdebe112994f999c69b336292163ed, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:00.731479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:00.746583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1109/2198] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:00 2022] PUT /identity/v3/projects/3568bcaeec69428ba4cb018cc8dd6a00/users/e7cdebe112994f999c69b336292163ed/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:00.756930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384555-a408-4716-8b1e-98a54d5424e8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:00.757266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384555-a408-4716-8b1e-98a54d5424e8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:00.757491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384555-a408-4716-8b1e-98a54d5424e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:00.803701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d8384555-a408-4716-8b1e-98a54d5424e8 None None] MFA Rules not processed for user `e7cdebe112994f999c69b336292163ed`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:00.807489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d8384555-a408-4716-8b1e-98a54d5424e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:00.900269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1090/2199] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:00.916354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:00.918219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:00.997977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878d128-a318-4f81-8a12-58c084aae353 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:01.005981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162ba02-171d-4e18-9fe1-ae2edc47b0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:01.006852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162ba02-171d-4e18-9fe1-ae2edc47b0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:01.011672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162ba02-171d-4e18-9fe1-ae2edc47b0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:01.014670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162ba02-171d-4e18-9fe1-ae2edc47b0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:01.030895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162ba02-171d-4e18-9fe1-ae2edc47b0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:01.032625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162ba02-171d-4e18-9fe1-ae2edc47b0b0 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:01.048322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1110/2200] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:02.015306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8384555-a408-4716-8b1e-98a54d5424e8 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:02.031254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8384555-a408-4716-8b1e-98a54d5424e8 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:02.035723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:02.036428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:02.036773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:02.037050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:02.158860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:02.160420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:02.182992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1091/2201] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:04.046182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c27d5-5461-4f10-acaa-e247b487f6f2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:04.046704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c27d5-5461-4f10-acaa-e247b487f6f2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:04.046784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c27d5-5461-4f10-acaa-e247b487f6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:04.086013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b39c27d5-5461-4f10-acaa-e247b487f6f2 None None] MFA Rules not processed for user `aeac5df40d57465e820d3b961633c5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:04.087421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b39c27d5-5461-4f10-acaa-e247b487f6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:04.183987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1111/2202] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:04.190817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:04.192354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:04.217932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:04.292585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2c10-2213-4244-b79b-4898951850cd service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:04.300291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:04.300291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:04.300291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:04.300925 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:04.302968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:04.374328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:04.376065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:04.388370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1092/2203] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:06.141615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261988c-df92-4409-989f-56d4232a2c84 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:06.142128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261988c-df92-4409-989f-56d4232a2c84 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:06.142244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261988c-df92-4409-989f-56d4232a2c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:06.166082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4261988c-df92-4409-989f-56d4232a2c84 None None] MFA Rules not processed for user `82fb5d585b5d460bb258f929057a4093`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:06.167481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4261988c-df92-4409-989f-56d4232a2c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:06.174910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1112/2204] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:06.851339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:06.853637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:06.865259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0030780-d28c-4508-9b28-226e154f6ab3 service nova] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:06.869923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:06.870536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:06.871520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:06.871798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:06.946751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:06.948340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:06.962901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1093/2205] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:07.125627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686461-c208-477a-83b4-0b9dbd388635 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:07.127309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686461-c208-477a-83b4-0b9dbd388635 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:07.127653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686461-c208-477a-83b4-0b9dbd388635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:07.154837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ef686461-c208-477a-83b4-0b9dbd388635 None None] MFA Rules not processed for user `ac0bc9e010d044a5a2147010cbdb4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:07.156389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ef686461-c208-477a-83b4-0b9dbd388635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:07.168956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:07.226411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1113/2206] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:09.895559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:09.975358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:09.993704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc98d-f036-4acf-bc15-f6b0440d361e service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:10.004182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:10.004884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:10.005339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:10.005619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] PATH_INFO: `/v3/users/328de220e50447a8809819a3d9b313ac` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:10.006775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=328de220e50447a8809819a3d9b313ac)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:10.009446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:10.080880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '328de220e50447a8809819a3d9b313ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.097533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '328de220e50447a8809819a3d9b313ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.098923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '328de220e50447a8809819a3d9b313ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.099821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '328de220e50447a8809819a3d9b313ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.108471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '328de220e50447a8809819a3d9b313ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.125707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1094/2207] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:09 2022] DELETE /identity/v3/users/328de220e50447a8809819a3d9b313ac => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:10.130284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:10.130770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:10.130848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:10.154265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:10.155690 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:10.222041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1114/2208] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:10:10.233371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:10.234844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:10.307318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d8552-16bc-436a-a04a-16e3da9022cc admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:10.311781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:10.312391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:10.312672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:10.313159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:10.331030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:10.332824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:10.345486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1095/2209] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:10.679392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:10.681212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:10.693927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb653-012d-41c7-9d1a-970249fa1ef4 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:10.699923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:10.700597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:10.700876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:10.701218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] PATH_INFO: `/v3/projects/7a1a864c665b495a910a93db3ed93b87` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:10.702359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1a864c665b495a910a93db3ed93b87)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:10.704339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:10.753551 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1a864c665b495a910a93db3ed93b87', 'request_id': 'req-4fc457fa-6949-4d66-9317-2b75d0be7015'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.760325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1a864c665b495a910a93db3ed93b87', 'request_id': 'req-4fc457fa-6949-4d66-9317-2b75d0be7015'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.766298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1a864c665b495a910a93db3ed93b87', 'request_id': 'req-4fc457fa-6949-4d66-9317-2b75d0be7015'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.780406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] The token cache is being invalidate because project 7a1a864c665b495a910a93db3ed93b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:10.780711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.781327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.782852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1115/2210] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:10 2022] DELETE /identity/v3/projects/7a1a864c665b495a910a93db3ed93b87 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:10.787233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:10.796905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdfc70-add0-43ae-8cf4-78d58593a386 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:10.866871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:10.868141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:10.868577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:10.868898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] PATH_INFO: `/v3/users/2d8017147b224017a837026cbab41135` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:10.870066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d8017147b224017a837026cbab41135)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:10.872164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:10.900006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:10.946667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d8017147b224017a837026cbab41135'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.975883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d8017147b224017a837026cbab41135'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.979311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d8017147b224017a837026cbab41135'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.980311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d8017147b224017a837026cbab41135'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:10.989226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d8017147b224017a837026cbab41135'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.020835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1096/2211] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:10 2022] DELETE /identity/v3/users/2d8017147b224017a837026cbab41135 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:11.544390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:11.546595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:11.567740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc457fa-6949-4d66-9317-2b75d0be7015 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:11.621110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:11.644454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:11.645108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:11.645390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:11.645663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] PATH_INFO: `/v3/projects/15e7858f3c154ec8a38c0069f83853a6` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:11.646971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e7858f3c154ec8a38c0069f83853a6)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:11.649837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:11.720471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e7858f3c154ec8a38c0069f83853a6', 'request_id': 'req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.728276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e7858f3c154ec8a38c0069f83853a6', 'request_id': 'req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.734113 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e7858f3c154ec8a38c0069f83853a6', 'request_id': 'req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.746988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] The token cache is being invalidate because project 15e7858f3c154ec8a38c0069f83853a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:11.747305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.748261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.750034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1116/2212] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:11 2022] DELETE /identity/v3/projects/15e7858f3c154ec8a38c0069f83853a6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:11.754073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:11.755543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:11.769291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb6fa5-280f-481c-92fc-0958cb707a11 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:11.841388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:11.842042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:11.842322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:11.842599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] PATH_INFO: `/v3/users/e7cdebe112994f999c69b336292163ed` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:11.843753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7cdebe112994f999c69b336292163ed)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:11.845675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:11.895410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7cdebe112994f999c69b336292163ed'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.907964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7cdebe112994f999c69b336292163ed'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.908888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7cdebe112994f999c69b336292163ed'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.909501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7cdebe112994f999c69b336292163ed'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.917541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-056538b9-70d8-46ce-980b-da3040b928e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7cdebe112994f999c69b336292163ed'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:11.932608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1097/2213] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:11 2022] DELETE /identity/v3/users/e7cdebe112994f999c69b336292163ed => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:12.237183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:12.238680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:12.256448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b503a1d-452c-41f4-aa84-a8e7c21b15cc admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:12.364736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:12.365919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.366440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.366931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] PATH_INFO: `/v3/projects/3568bcaeec69428ba4cb018cc8dd6a00` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.369241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3568bcaeec69428ba4cb018cc8dd6a00)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:12.371480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:12.442739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3568bcaeec69428ba4cb018cc8dd6a00', 'request_id': 'req-00210810-bffc-4925-ab88-d1d8f60cbb51'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:12.450735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3568bcaeec69428ba4cb018cc8dd6a00', 'request_id': 'req-00210810-bffc-4925-ab88-d1d8f60cbb51'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:12.480770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3568bcaeec69428ba4cb018cc8dd6a00', 'request_id': 'req-00210810-bffc-4925-ab88-d1d8f60cbb51'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:12.506271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] The token cache is being invalidate because project 3568bcaeec69428ba4cb018cc8dd6a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:12.506837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:12.507801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-00210810-bffc-4925-ab88-d1d8f60cbb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:12.512436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1117/2214] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:12 2022] DELETE /identity/v3/projects/3568bcaeec69428ba4cb018cc8dd6a00 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:12.519975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fa690-e3b2-4277-98af-6cf555f1acf5 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.520685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fa690-e3b2-4277-98af-6cf555f1acf5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.521435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fa690-e3b2-4277-98af-6cf555f1acf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.557916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-556fa690-e3b2-4277-98af-6cf555f1acf5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:12.559515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-556fa690-e3b2-4277-98af-6cf555f1acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:12.566600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:12.644536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1098/2215] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:10:12.654381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e363b21-1a58-4e07-b89c-14a68288febc None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.654998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e363b21-1a58-4e07-b89c-14a68288febc None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.655135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e363b21-1a58-4e07-b89c-14a68288febc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.666674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:12.681988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7e363b21-1a58-4e07-b89c-14a68288febc None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:12.683529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7e363b21-1a58-4e07-b89c-14a68288febc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:12.692379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1118/2216] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:12.697201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fa690-e3b2-4277-98af-6cf555f1acf5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:12.708108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fa690-e3b2-4277-98af-6cf555f1acf5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:12.713076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:12.713731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.714007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.714284 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.714948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:10:12.715280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:12.716904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:12.735214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1099/2217] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:10:12.739410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e363b21-1a58-4e07-b89c-14a68288febc None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:12.750732 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e363b21-1a58-4e07-b89c-14a68288febc None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:12.764808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:12.765425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.765702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.766000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.766665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:12.768731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:12.800362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1119/2218] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:10:12 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:12.804777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:12.817325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7b0b-6948-4949-b19c-f66b108a7eb5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:12.880792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:12.881408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.881693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.881966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.882608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:12.884916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:12.916788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1100/2219] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:10:12 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:12.917947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:12.931444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7840bd8-b1d7-4a7f-95ee-31d44bf23db4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:12.940403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:12.941344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.941805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.942225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.943244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:12.945146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:12.966290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1120/2220] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:12.966992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:12.978776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d6be9-cfde-4d53-9a99-57d929474d57 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:12.985663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fc9d0-d620-458d-b6f3-5408abdb6192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:12.986370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fc9d0-d620-458d-b6f3-5408abdb6192 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:12.986730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fc9d0-d620-458d-b6f3-5408abdb6192 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:12.986994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fc9d0-d620-458d-b6f3-5408abdb6192 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:12.987768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fc9d0-d620-458d-b6f3-5408abdb6192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:12.989286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fc9d0-d620-458d-b6f3-5408abdb6192 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.005709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1101/2221] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:13.007982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.020721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f375e8-4b06-42fc-b36b-81d98c68b1ba admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:13.027832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:13.028973 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.029245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.029931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] PATH_INFO: `/v3/projects/6a7863bb2e38467db832099c3edc7e96/users/e3b7e46f2c6f4691956bfc0b9c264ea4/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.058336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7863bb2e38467db832099c3edc7e96, user_id=e3b7e46f2c6f4691956bfc0b9c264ea4, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:13.061758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.110806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1121/2222] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:13 2022] PUT /identity/v3/projects/6a7863bb2e38467db832099c3edc7e96/users/e3b7e46f2c6f4691956bfc0b9c264ea4/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:13.122937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fadfe-68ac-4fb5-99cb-b3ab3c6b8552 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.123316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fadfe-68ac-4fb5-99cb-b3ab3c6b8552 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.123527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fadfe-68ac-4fb5-99cb-b3ab3c6b8552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.172845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a18fadfe-68ac-4fb5-99cb-b3ab3c6b8552 None None] MFA Rules not processed for user `e3b7e46f2c6f4691956bfc0b9c264ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:13.174186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a18fadfe-68ac-4fb5-99cb-b3ab3c6b8552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:13.289400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1102/2223] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:13.291977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.369353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b90a71-3ff4-4d6e-927c-5592bdce6673 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:13.375155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:13.375821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.376103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.376438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.377006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:13.378914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.411442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1122/2224] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:10:13 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:10:13.412747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fadfe-68ac-4fb5-99cb-b3ab3c6b8552 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.424798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fadfe-68ac-4fb5-99cb-b3ab3c6b8552 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:13.428934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:13.489470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:13.490009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.490285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.490649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.491339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:13.493627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.519332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1103/2225] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:10:13 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:13.520708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.531807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45041011-e575-4de4-a356-8a16ed128d47 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:13.537045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:13.537814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.538252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.538607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.539514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:13.541706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.558295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1123/2226] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:10:13.562812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.590653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56be08-9d28-4128-b5c3-79cba4100dd4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:13.594670 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:13.595380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.595663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.595938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] PATH_INFO: `/v3/projects/050735b3e67d4e19b4ac096774423a4c/users/b173e1c6b88244f98767edf5a574e84a/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.622919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=050735b3e67d4e19b4ac096774423a4c, user_id=b173e1c6b88244f98767edf5a574e84a, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:13.625698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.643102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1104/2227] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:13 2022] PUT /identity/v3/projects/050735b3e67d4e19b4ac096774423a4c/users/b173e1c6b88244f98767edf5a574e84a/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:13.647209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.659251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61976ea6-3e72-4594-89ef-0c046718dc88 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:13.720384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c798537-2867-420d-b28d-eaa657aa26b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:13.721033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798537-2867-420d-b28d-eaa657aa26b9 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.721331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798537-2867-420d-b28d-eaa657aa26b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.721605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798537-2867-420d-b28d-eaa657aa26b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.722248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c798537-2867-420d-b28d-eaa657aa26b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:13.723880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c798537-2867-420d-b28d-eaa657aa26b9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.734361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1124/2228] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:13.738746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.749596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc403-c186-4322-84f2-0cc407981b14 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:13.758770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:13.759940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.760438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.760875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] PATH_INFO: `/v3/projects/050735b3e67d4e19b4ac096774423a4c/users/b173e1c6b88244f98767edf5a574e84a/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.762847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=050735b3e67d4e19b4ac096774423a4c, user_id=b173e1c6b88244f98767edf5a574e84a, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:13.765530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:13.777504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1105/2229] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:13 2022] PUT /identity/v3/projects/050735b3e67d4e19b4ac096774423a4c/users/b173e1c6b88244f98767edf5a574e84a/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:10:13.788284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5acbc-511e-4aea-a6c6-7b0f5b8fcd80 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:13.788574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5acbc-511e-4aea-a6c6-7b0f5b8fcd80 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:13.788852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5acbc-511e-4aea-a6c6-7b0f5b8fcd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:13.812486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:13.825582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8ba5acbc-511e-4aea-a6c6-7b0f5b8fcd80 None None] MFA Rules not processed for user `b173e1c6b88244f98767edf5a574e84a`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:13.827766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8ba5acbc-511e-4aea-a6c6-7b0f5b8fcd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:13.907163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1125/2230] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:13.919531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:13.922786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:14.000150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03718c-9ab2-4600-ac3a-93da07e020a8 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:14.004294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:14.005224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:14.005224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:14.005418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:14.016735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:14.018167 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:14.028991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1106/2231] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:14.140487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5acbc-511e-4aea-a6c6-7b0f5b8fcd80 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:14.153494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5acbc-511e-4aea-a6c6-7b0f5b8fcd80 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:14.157747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595db33-767d-4c38-ab92-9fa8d867740c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:14.158385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595db33-767d-4c38-ab92-9fa8d867740c service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:14.158968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595db33-767d-4c38-ab92-9fa8d867740c service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:14.159334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595db33-767d-4c38-ab92-9fa8d867740c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:14.237024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595db33-767d-4c38-ab92-9fa8d867740c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:14.238569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595db33-767d-4c38-ab92-9fa8d867740c service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:14.249742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1126/2232] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:14.653986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:14.759843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdc2ca-1ca6-4581-9e34-4ad3ef96eb59 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:14.764501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:14.765137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:14.765422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:14.765697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] PATH_INFO: `/v3/users/e3b7e46f2c6f4691956bfc0b9c264ea4` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:14.766832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b7e46f2c6f4691956bfc0b9c264ea4)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:14.768866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:14.771456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:14.828946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b7e46f2c6f4691956bfc0b9c264ea4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:14.845325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b7e46f2c6f4691956bfc0b9c264ea4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:14.846285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b7e46f2c6f4691956bfc0b9c264ea4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:14.847169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b7e46f2c6f4691956bfc0b9c264ea4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:14.858948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b7e46f2c6f4691956bfc0b9c264ea4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:14.884400 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:14.885505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:14.886034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:14.888999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1107/2233] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:14 2022] DELETE /identity/v3/users/e3b7e46f2c6f4691956bfc0b9c264ea4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:14.911120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:14.913494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:14.999312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1127/2234] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:15.009629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:15.011024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:15.089341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e19bec-650b-4377-bfea-c0f51fea5ad0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:15.093287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:15.094358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:15.094857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:15.095171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:15.105080 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:15.106523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:15.117681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1108/2235] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:15.438876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:15.442532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:15.462763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4af9-0d3e-4ad2-a2a3-4946d9e309eb None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:15.471532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:15.472228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:15.474860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:15.474967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] PATH_INFO: `/v3/projects/6a7863bb2e38467db832099c3edc7e96` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:15.483623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7863bb2e38467db832099c3edc7e96)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:15.486016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:15.531126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:15.554001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7863bb2e38467db832099c3edc7e96', 'request_id': 'req-95a88fa1-22a5-461f-8c96-2296768ba7df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.561576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7863bb2e38467db832099c3edc7e96', 'request_id': 'req-95a88fa1-22a5-461f-8c96-2296768ba7df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.567980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7863bb2e38467db832099c3edc7e96', 'request_id': 'req-95a88fa1-22a5-461f-8c96-2296768ba7df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.583704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] The token cache is being invalidate because project 6a7863bb2e38467db832099c3edc7e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:15.583998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.585269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.587333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1128/2236] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:15 2022] DELETE /identity/v3/projects/6a7863bb2e38467db832099c3edc7e96 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:15.593637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:15.606951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545167a-baee-43e6-b3da-a974ff909719 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:15.712320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:15.713107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:15.713389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:15.713663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] PATH_INFO: `/v3/users/b173e1c6b88244f98767edf5a574e84a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:15.715485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b173e1c6b88244f98767edf5a574e84a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:15.718009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:15.758694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:15.788681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b173e1c6b88244f98767edf5a574e84a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.802007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b173e1c6b88244f98767edf5a574e84a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.803013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b173e1c6b88244f98767edf5a574e84a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.804020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b173e1c6b88244f98767edf5a574e84a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.813038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a20c2692-4e7e-4be4-a75f-daa8a061591d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b173e1c6b88244f98767edf5a574e84a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:15.826234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1109/2237] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:15 2022] DELETE /identity/v3/users/b173e1c6b88244f98767edf5a574e84a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:16.170783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.172580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:16.195475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88fa1-22a5-461f-8c96-2296768ba7df admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.274572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.275314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.275908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.276200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] PATH_INFO: `/v3/projects/050735b3e67d4e19b4ac096774423a4c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.277415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=050735b3e67d4e19b4ac096774423a4c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.279399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.333475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '050735b3e67d4e19b4ac096774423a4c', 'request_id': 'req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:16.342037 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '050735b3e67d4e19b4ac096774423a4c', 'request_id': 'req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:16.348155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '050735b3e67d4e19b4ac096774423a4c', 'request_id': 'req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:16.362160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] The token cache is being invalidate because project 050735b3e67d4e19b4ac096774423a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:16.362455 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:16.363090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8d47e12d-98ba-4e13-9f53-b5a8619b84c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:16.364600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1129/2238] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:16 2022] DELETE /identity/v3/projects/050735b3e67d4e19b4ac096774423a4c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:16.369612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81999035-636d-4ab7-b06c-a495da32867f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.370094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81999035-636d-4ab7-b06c-a495da32867f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.370205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81999035-636d-4ab7-b06c-a495da32867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.389897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-81999035-636d-4ab7-b06c-a495da32867f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:16.391121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-81999035-636d-4ab7-b06c-a495da32867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:16.452165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1110/2239] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:16.460041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689577c-7efd-47c5-a13f-d094d8fd05b5 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.460331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689577c-7efd-47c5-a13f-d094d8fd05b5 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.460638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689577c-7efd-47c5-a13f-d094d8fd05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.483359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-5689577c-7efd-47c5-a13f-d094d8fd05b5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:16.484589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-5689577c-7efd-47c5-a13f-d094d8fd05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:16.492394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1130/2240] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:16.496862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81999035-636d-4ab7-b06c-a495da32867f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.506906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81999035-636d-4ab7-b06c-a495da32867f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.512774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.513382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.513656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.513924 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.514577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:10:16.514876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.516490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.529905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1111/2241] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:10:16.534419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689577c-7efd-47c5-a13f-d094d8fd05b5 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.544039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689577c-7efd-47c5-a13f-d094d8fd05b5 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.548675 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.549295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.549569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.549834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.550462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.552366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.582003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1131/2242] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:10:16 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:16.585895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.596889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7dfb8-7014-4634-852d-2eecd3080c87 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.653966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.654584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.654870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.655201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.655870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.657918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.682713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1112/2243] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:10:16 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:10:16.686850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.698151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91015fdf-30ab-45ac-988d-4aa35fa16461 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.702987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.703618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.703898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.704175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.705344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.706393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.720718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1132/2244] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:10:16.722211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.734474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0cf01-1a9e-4f92-b0f6-c5ff2560c255 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.739360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e2655-7d3b-4026-ac15-32e2b35f7ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.740050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e2655-7d3b-4026-ac15-32e2b35f7ddb admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.740317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e2655-7d3b-4026-ac15-32e2b35f7ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.740512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e2655-7d3b-4026-ac15-32e2b35f7ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.741162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e2655-7d3b-4026-ac15-32e2b35f7ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.742896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e2655-7d3b-4026-ac15-32e2b35f7ddb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.755002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1113/2245] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:10:16.759834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:16.771905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b6701-fda1-4dc8-bdd8-1216d79a788a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:16.777833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:16.778516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.778798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.779111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] PATH_INFO: `/v3/projects/49624da95b924b9ebf10ea073c1f2daa/users/945ec6dcc1644e63879d3e8333cce0bc/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.782498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:16.827158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49624da95b924b9ebf10ea073c1f2daa, user_id=945ec6dcc1644e63879d3e8333cce0bc, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:16.835343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:16.868226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1133/2246] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:10:16 2022] PUT /identity/v3/projects/49624da95b924b9ebf10ea073c1f2daa/users/945ec6dcc1644e63879d3e8333cce0bc/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:16.878454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a239-5334-4090-97e1-116c86aeab41 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:16.878771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a239-5334-4090-97e1-116c86aeab41 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:16.879025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a239-5334-4090-97e1-116c86aeab41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:16.903131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:16.916825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b494a239-5334-4090-97e1-116c86aeab41 None None] MFA Rules not processed for user `945ec6dcc1644e63879d3e8333cce0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:16.918758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b494a239-5334-4090-97e1-116c86aeab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:16.992822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1114/2247] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:17.004612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:17.006250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:17.065014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a34b48-fa94-4372-97f2-a471872359f6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:17.068652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:17.069255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:17.069547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:17.069815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:17.078120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:17.079800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:17.091188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1134/2248] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:20.237539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18a0c4-2479-4b76-b531-3880ee1a82bc None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:20.237994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18a0c4-2479-4b76-b531-3880ee1a82bc None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:20.238181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18a0c4-2479-4b76-b531-3880ee1a82bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:20.258468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5e18a0c4-2479-4b76-b531-3880ee1a82bc None None] MFA Rules not processed for user `73c4a7ab7fc7425389843798faa25372`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:20.260590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e18a0c4-2479-4b76-b531-3880ee1a82bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:20.320010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1115/2249] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:21.324843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:21.407863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bbb32-f67e-46d0-8c5b-95190a37e3de service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:21.413804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:21.414628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:21.415058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:21.416087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] PATH_INFO: `/v3/users/945ec6dcc1644e63879d3e8333cce0bc` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:21.419419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] RBAC: Authorizing `identity:delete_user(user_id=945ec6dcc1644e63879d3e8333cce0bc)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:21.419419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:21.501663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945ec6dcc1644e63879d3e8333cce0bc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:21.512790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945ec6dcc1644e63879d3e8333cce0bc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:21.513672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945ec6dcc1644e63879d3e8333cce0bc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:21.528881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945ec6dcc1644e63879d3e8333cce0bc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:21.537151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945ec6dcc1644e63879d3e8333cce0bc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:21.548303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1135/2250] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:10:21 2022] DELETE /identity/v3/users/945ec6dcc1644e63879d3e8333cce0bc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:10:21.549870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:21.550329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:21.550575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:21.573794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:21.574995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:21.642660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1116/2251] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:21.654679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:21.656149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:21.668648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:21.728802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190fe4-ece9-4ab2-999f-5c64e32f487a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:21.732680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879a7cc-f450-4f4a-9b6d-2767002d6378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:21.733292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879a7cc-f450-4f4a-9b6d-2767002d6378 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:21.733601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879a7cc-f450-4f4a-9b6d-2767002d6378 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:21.733891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879a7cc-f450-4f4a-9b6d-2767002d6378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:21.745178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879a7cc-f450-4f4a-9b6d-2767002d6378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:21.746724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879a7cc-f450-4f4a-9b6d-2767002d6378 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:21.758062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1136/2252] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:22.042448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:22.044131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:22.057854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c322-eabc-4683-8145-f70e7b8a1041 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:22.062673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:22.063339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:22.063801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:22.064234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] PATH_INFO: `/v3/projects/49624da95b924b9ebf10ea073c1f2daa` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:22.065946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49624da95b924b9ebf10ea073c1f2daa)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:22.069885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:22.126021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:22.143180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49624da95b924b9ebf10ea073c1f2daa', 'request_id': 'req-d2d8df7f-5f7b-4154-9caa-97198a588783'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:22.151373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49624da95b924b9ebf10ea073c1f2daa', 'request_id': 'req-d2d8df7f-5f7b-4154-9caa-97198a588783'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:22.160562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49624da95b924b9ebf10ea073c1f2daa', 'request_id': 'req-d2d8df7f-5f7b-4154-9caa-97198a588783'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:22.177348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] The token cache is being invalidate because project 49624da95b924b9ebf10ea073c1f2daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:10:22.177656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:22.178309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d2d8df7f-5f7b-4154-9caa-97198a588783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:10:22.180183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1117/2253] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:10:22 2022] DELETE /identity/v3/projects/49624da95b924b9ebf10ea073c1f2daa => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:10:23.987344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428cfcc-9000-442e-b51b-020644746961 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:23.987861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428cfcc-9000-442e-b51b-020644746961 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:23.988665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428cfcc-9000-442e-b51b-020644746961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:24.010614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1428cfcc-9000-442e-b51b-020644746961 None None] MFA Rules not processed for user `8f1c0b2189a041fdb8dbe2a0290ffac8`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:24.011768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1428cfcc-9000-442e-b51b-020644746961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:24.081413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1137/2254] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:10:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:29.849807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:29.850277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:29.850394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:29.875570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:29.876889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:29.957931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1118/2255] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:10:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:29.974467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428cfcc-9000-442e-b51b-020644746961 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:29.976137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1428cfcc-9000-442e-b51b-020644746961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:30.122521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428cfcc-9000-442e-b51b-020644746961 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:30.127616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ed3bd-bb97-4163-90e5-a06133fff689 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:30.128240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed3bd-bb97-4163-90e5-a06133fff689 service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:30.128573 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed3bd-bb97-4163-90e5-a06133fff689 service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:30.128993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed3bd-bb97-4163-90e5-a06133fff689 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:30.138162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ed3bd-bb97-4163-90e5-a06133fff689 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:30.140157 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ed3bd-bb97-4163-90e5-a06133fff689 service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:30.148841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1138/2256] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:10:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:10:39.946882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=35,keystone:DELETE=4,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:40.145867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=80 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:53.997385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:54.003347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:54.087420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e5027-c79b-47c5-b2b5-0c4a626ffe58 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:54.094278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2386735-cd6a-491a-81d9-8f847e45e8e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:54.094472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2386735-cd6a-491a-81d9-8f847e45e8e7 service placement] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:54.094754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2386735-cd6a-491a-81d9-8f847e45e8e7 service placement] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:54.095027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2386735-cd6a-491a-81d9-8f847e45e8e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:54.096833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a2386735-cd6a-491a-81d9-8f847e45e8e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:54.108781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2386735-cd6a-491a-81d9-8f847e45e8e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:54.110370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2386735-cd6a-491a-81d9-8f847e45e8e7 service placement] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:54.121016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1119/2257] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:55.398201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c7bae-99f6-4929-81b2-84b0d89b7f06 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.398860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c7bae-99f6-4929-81b2-84b0d89b7f06 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.398860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c7bae-99f6-4929-81b2-84b0d89b7f06 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.400143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1139/2258] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:10:55.409153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b037-aac6-4dac-82ec-042f9b08c897 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.409618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b037-aac6-4dac-82ec-042f9b08c897 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.409732 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b037-aac6-4dac-82ec-042f9b08c897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.431332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8517b037-aac6-4dac-82ec-042f9b08c897 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:55.432883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8517b037-aac6-4dac-82ec-042f9b08c897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:55.514832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1120/2259] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:10:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:55.526986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c7bae-99f6-4929-81b2-84b0d89b7f06 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:55.530051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e55c7bae-99f6-4929-81b2-84b0d89b7f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:55.592362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c7bae-99f6-4929-81b2-84b0d89b7f06 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:55.595868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb69bb-02ca-4996-87b4-0509c93cac16 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:55.596470 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb69bb-02ca-4996-87b4-0509c93cac16 service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.596749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb69bb-02ca-4996-87b4-0509c93cac16 service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.597019 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb69bb-02ca-4996-87b4-0509c93cac16 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.608638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb69bb-02ca-4996-87b4-0509c93cac16 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:55.610135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb69bb-02ca-4996-87b4-0509c93cac16 service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:55.621045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1140/2260] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:55.691467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f90ff-d5ea-4adc-882f-34eeb537e4fc None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.691788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f90ff-d5ea-4adc-882f-34eeb537e4fc None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.692062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f90ff-d5ea-4adc-882f-34eeb537e4fc None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.693401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1121/2261] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:10:55.698120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f141-0b17-44c8-82fd-214698d83a34 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.698403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f141-0b17-44c8-82fd-214698d83a34 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.698693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f141-0b17-44c8-82fd-214698d83a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.719384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8320f141-0b17-44c8-82fd-214698d83a34 None None] MFA Rules not processed for user `70c6682b92694484b2fa2710314f547f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:55.720683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8320f141-0b17-44c8-82fd-214698d83a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:55.771344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1141/2262] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:10:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:10:55.786468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f90ff-d5ea-4adc-882f-34eeb537e4fc None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:55.788243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9d8f90ff-d5ea-4adc-882f-34eeb537e4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:55.802509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f90ff-d5ea-4adc-882f-34eeb537e4fc None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:55.806234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c1847-562e-4f3c-88c5-303ad8f5072d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:55.806838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c1847-562e-4f3c-88c5-303ad8f5072d service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.807148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c1847-562e-4f3c-88c5-303ad8f5072d service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.807426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c1847-562e-4f3c-88c5-303ad8f5072d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.815093 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c1847-562e-4f3c-88c5-303ad8f5072d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:55.816605 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c1847-562e-4f3c-88c5-303ad8f5072d service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:55.824804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1122/2263] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:55.911129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e1564-d423-493e-89ee-955d815de139 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.911426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e1564-d423-493e-89ee-955d815de139 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.911708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e1564-d423-493e-89ee-955d815de139 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.913213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1142/2264] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:10:55.919123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec14ce6-1f9f-49eb-8492-65bc4444bb73 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:55.920653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec14ce6-1f9f-49eb-8492-65bc4444bb73 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:55.920653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec14ce6-1f9f-49eb-8492-65bc4444bb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:55.942878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bec14ce6-1f9f-49eb-8492-65bc4444bb73 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:10:55.944203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bec14ce6-1f9f-49eb-8492-65bc4444bb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:55.954514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1123/2265] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:10:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:10:56.004465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e1564-d423-493e-89ee-955d815de139 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:56.006318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a07e1564-d423-493e-89ee-955d815de139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:56.024028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e1564-d423-493e-89ee-955d815de139 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:56.028944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabae93f-3e3b-44b1-be1d-e9690113bc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:56.030904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabae93f-3e3b-44b1-be1d-e9690113bc7f service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:56.031299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabae93f-3e3b-44b1-be1d-e9690113bc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:56.031654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabae93f-3e3b-44b1-be1d-e9690113bc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:56.046126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabae93f-3e3b-44b1-be1d-e9690113bc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:56.047714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabae93f-3e3b-44b1-be1d-e9690113bc7f service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:56.060409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1143/2266] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:10:58.465160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec14ce6-1f9f-49eb-8492-65bc4444bb73 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:10:58.475771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec14ce6-1f9f-49eb-8492-65bc4444bb73 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:10:58.479319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:10:58.480500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:10:58.480976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:10:58.481408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:10:58.483688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:10:58.548812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:10:58.551295 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:10:58.553248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:10:58.564106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1124/2267] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:10:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:06.055356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=55 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:08.559226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:10.274298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f9004-0e6c-4766-937a-3376d7120090 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:10.274796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f9004-0e6c-4766-937a-3376d7120090 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:10.274901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f9004-0e6c-4766-937a-3376d7120090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:10.293978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c21f9004-0e6c-4766-937a-3376d7120090 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:10.295210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c21f9004-0e6c-4766-937a-3376d7120090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:10.302996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1144/2268] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:11:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:10.315498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:10.327174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521229d-8394-48a3-8a86-d5fb0750460e service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:10.332992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:10.333557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:10.333839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:10.334113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:10.342482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:10.344413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:10.352698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1125/2269] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:11:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:11:15.419010 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e9265-2801-4f6b-be3a-2aabe99aa4de None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:15.419738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e9265-2801-4f6b-be3a-2aabe99aa4de None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:15.419738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e9265-2801-4f6b-be3a-2aabe99aa4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:15.439579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-654e9265-2801-4f6b-be3a-2aabe99aa4de None None] MFA Rules not processed for user `985dfcc7246c45b5b470e07957cd2f49`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:15.440814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-654e9265-2801-4f6b-be3a-2aabe99aa4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:15.499962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1145/2270] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:11:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:20.349325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=6 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:25.094023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:25.103315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e748248-3c8b-4fed-8cfa-ccfea84830ac service swift] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:25.106892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:25.107514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:25.107792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:25.108063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:25.110044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:25.173142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:25.174522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:25.183997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1126/2271] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:11:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:11:25.492892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=28 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:35.169117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab024892-043a-44de-88cd-196e7e999d7f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:35.169674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab024892-043a-44de-88cd-196e7e999d7f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:35.169778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab024892-043a-44de-88cd-196e7e999d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:35.179701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=30 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:35.190799 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ab024892-043a-44de-88cd-196e7e999d7f None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:11:35.192067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ab024892-043a-44de-88cd-196e7e999d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:11:35.199209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1146/2272] 10.222.0.94 () {58 vars in 967 bytes} [Mon Aug 29 19:11:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:11:35.212146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:11:35.223741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e55ef9-87a8-4c0a-b2ae-23333124478a service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:11:35.228817 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd152be-756e-489f-a868-1426d22dfd96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:11:35.229428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd152be-756e-489f-a868-1426d22dfd96 service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:11:35.229709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd152be-756e-489f-a868-1426d22dfd96 service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:11:35.229987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd152be-756e-489f-a868-1426d22dfd96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:11:35.238899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd152be-756e-489f-a868-1426d22dfd96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:11:35.240512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd152be-756e-489f-a868-1426d22dfd96 service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:11:35.248591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1127/2273] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:11:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:11:45.185238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:11:45.245334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=6 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:08.025029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b481-4b6f-4c8c-a4ef-40a569740433 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:08.025604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b481-4b6f-4c8c-a4ef-40a569740433 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:08.025604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b481-4b6f-4c8c-a4ef-40a569740433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:08.046202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c298b481-4b6f-4c8c-a4ef-40a569740433 None None] MFA Rules not processed for user `ac0bc9e010d044a5a2147010cbdb4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:08.047431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c298b481-4b6f-4c8c-a4ef-40a569740433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:08.102270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1147/2274] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:12:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:18.095268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=25 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:37.756667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:37.757247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:37.757247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:37.776334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] MFA Rules not processed for user `73c4a7ab7fc7425389843798faa25372`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:37.778131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:37.833749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1128/2275] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:12:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:42.222789 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298b481-4b6f-4c8c-a4ef-40a569740433 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:42.235594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298b481-4b6f-4c8c-a4ef-40a569740433 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:42.240957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:42.241672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:42.242047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:42.242422 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:42.244274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:42.258813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:42.260479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:42.272569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1148/2276] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:12:42.302196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:42.304053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:42.380316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c78e1-3d86-464c-9722-e4cb61b7fdc9 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:42.384882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:42.385498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:42.385776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:42.386045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] PATH_INFO: `/v3/users/985dfcc7246c45b5b470e07957cd2f49` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:42.388214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985dfcc7246c45b5b470e07957cd2f49)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:42.390119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:42.436966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985dfcc7246c45b5b470e07957cd2f49'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.449844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985dfcc7246c45b5b470e07957cd2f49'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.450703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985dfcc7246c45b5b470e07957cd2f49'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.451370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985dfcc7246c45b5b470e07957cd2f49'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.458184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f56e6d8f-09ad-455e-af95-1840d6410ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985dfcc7246c45b5b470e07957cd2f49'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.471666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1129/2277] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:12:42 2022] DELETE /identity/v3/users/985dfcc7246c45b5b470e07957cd2f49 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:12:42.650826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:42.662513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116501ef-f42b-42e0-85e6-8fe0495f68a6 service placement] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:42.723538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:42.724176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:42.724456 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:42.724724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] PATH_INFO: `/v3/projects/778d0dad05594da5bb3cb0e2efcb2af5` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:42.725929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778d0dad05594da5bb3cb0e2efcb2af5)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:42.727932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:42.777460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778d0dad05594da5bb3cb0e2efcb2af5', 'request_id': 'req-84750ecb-6e08-49ac-a310-e78d65f92654'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.784043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778d0dad05594da5bb3cb0e2efcb2af5', 'request_id': 'req-84750ecb-6e08-49ac-a310-e78d65f92654'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.789487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778d0dad05594da5bb3cb0e2efcb2af5', 'request_id': 'req-84750ecb-6e08-49ac-a310-e78d65f92654'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.805053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] The token cache is being invalidate because project 778d0dad05594da5bb3cb0e2efcb2af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:12:42.805350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.806128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-84750ecb-6e08-49ac-a310-e78d65f92654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:42.808069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1149/2278] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:12:42 2022] DELETE /identity/v3/projects/778d0dad05594da5bb3cb0e2efcb2af5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:12:42.813315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175576-5dee-4e56-9a57-6cd63a82c4cd None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:42.813997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175576-5dee-4e56-9a57-6cd63a82c4cd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:42.814423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175576-5dee-4e56-9a57-6cd63a82c4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:42.838922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-2e175576-5dee-4e56-9a57-6cd63a82c4cd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:42.840539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2e175576-5dee-4e56-9a57-6cd63a82c4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:42.863836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:42.908297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1130/2279] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:42.916251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea118-b6e6-47c8-96b8-866336d2d533 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:42.916539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea118-b6e6-47c8-96b8-866336d2d533 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:42.916814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea118-b6e6-47c8-96b8-866336d2d533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:42.938794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4a5ea118-b6e6-47c8-96b8-866336d2d533 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:42.940083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4a5ea118-b6e6-47c8-96b8-866336d2d533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:42.948233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1150/2280] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:42.952664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175576-5dee-4e56-9a57-6cd63a82c4cd None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:42.962651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175576-5dee-4e56-9a57-6cd63a82c4cd None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:42.967610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:42.968218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:42.968499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:42.968774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:42.969427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:12:42.969730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:42.971502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:42.990465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1131/2281] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:12:42.991288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ea118-b6e6-47c8-96b8-866336d2d533 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:43.002555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ea118-b6e6-47c8-96b8-866336d2d533 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:43.007246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:43.007866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.008127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.008399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.009046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:43.010944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:43.039189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1151/2282] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:12:42 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:12:43.044196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:43.054603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b5617-7fe5-4455-9236-4e92b90bc3e1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:43.111798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:43.112337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.112603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.112875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.113515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:43.115458 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:43.136686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1132/2283] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:12:43 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:12:43.141235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:43.151625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e275e5-6dae-4600-84b0-aaa98dee1b29 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:43.157077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:43.157678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.157961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.158237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.158877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:43.160522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:43.171541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1152/2284] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:12:43.175678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:43.185679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c078af9-cdd4-48bb-9c95-2877976b55f7 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:43.190395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b2c06-01c4-415e-8601-c4c1051fc709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:43.190986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2c06-01c4-415e-8601-c4c1051fc709 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.191296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2c06-01c4-415e-8601-c4c1051fc709 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.191570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2c06-01c4-415e-8601-c4c1051fc709 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.192215 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b2c06-01c4-415e-8601-c4c1051fc709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:43.193803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b2c06-01c4-415e-8601-c4c1051fc709 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:43.204827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1133/2285] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:12:43.208987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:43.219447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d8495-e313-480e-a95d-7d75e0935bc9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:43.224149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:43.224797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.225076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.225346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] PATH_INFO: `/v3/projects/74c4e0da86aa42e1b5062dab2a174b18/users/65e16412487f4cbc862db8947e6ab1d7/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.240893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c4e0da86aa42e1b5062dab2a174b18, user_id=65e16412487f4cbc862db8947e6ab1d7, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:43.243809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:43.257228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1153/2286] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:12:43 2022] PUT /identity/v3/projects/74c4e0da86aa42e1b5062dab2a174b18/users/65e16412487f4cbc862db8947e6ab1d7/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:12:43.268675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becbfe2-9b28-488b-a463-6307e63a54ae None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.268995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becbfe2-9b28-488b-a463-6307e63a54ae None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.269300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becbfe2-9b28-488b-a463-6307e63a54ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.298875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8becbfe2-9b28-488b-a463-6307e63a54ae None None] MFA Rules not processed for user `65e16412487f4cbc862db8947e6ab1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:43.300124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8becbfe2-9b28-488b-a463-6307e63a54ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:43.367999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1134/2287] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:12:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:43.379544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:43.381343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:43.396499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:43.448115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81540f92-bbaf-47fc-b6bd-e1543846fef0 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:43.452550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:43.453163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:43.453445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:43.453726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:43.462376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:43.463927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:43.473636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1154/2288] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:51.120265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab34c-89e3-4837-9589-ee8cea0b7ff8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:51.120786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab34c-89e3-4837-9589-ee8cea0b7ff8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:51.120901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab34c-89e3-4837-9589-ee8cea0b7ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:51.140773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cf0ab34c-89e3-4837-9589-ee8cea0b7ff8 None None] MFA Rules not processed for user `ff8ce50a95b742a48500e634db50fb36`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:51.141992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cf0ab34c-89e3-4837-9589-ee8cea0b7ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:51.155275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:51.199784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1135/2289] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:12:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:51.238783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:51.240349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:51.300476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2e89-f984-4369-8973-415cec352810 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:51.304197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979acd7-6a45-481e-8913-e4c1319fdd38 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:51.304800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979acd7-6a45-481e-8913-e4c1319fdd38 service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:51.305097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979acd7-6a45-481e-8913-e4c1319fdd38 service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:51.305384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979acd7-6a45-481e-8913-e4c1319fdd38 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:51.307139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3979acd7-6a45-481e-8913-e4c1319fdd38 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:51.379918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979acd7-6a45-481e-8913-e4c1319fdd38 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:51.381599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979acd7-6a45-481e-8913-e4c1319fdd38 service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:51.393062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1155/2290] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:51.732916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ab34c-89e3-4837-9589-ee8cea0b7ff8 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:51.802532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ab34c-89e3-4837-9589-ee8cea0b7ff8 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:51.807620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:51.808285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:51.808565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:51.809645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] PATH_INFO: `/v3/users/65e16412487f4cbc862db8947e6ab1d7` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:51.811020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e16412487f4cbc862db8947e6ab1d7)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:51.812964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:51.864506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e16412487f4cbc862db8947e6ab1d7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:51.878187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e16412487f4cbc862db8947e6ab1d7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:51.878902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e16412487f4cbc862db8947e6ab1d7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:51.879623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e16412487f4cbc862db8947e6ab1d7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:51.886591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e16412487f4cbc862db8947e6ab1d7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:51.901253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1136/2291] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:12:51 2022] DELETE /identity/v3/users/65e16412487f4cbc862db8947e6ab1d7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:12:51.906156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:51.906507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:51.906845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:51.927837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:12:51.929034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:51.998207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:52.008248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1156/2292] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:12:52.018825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:52.020950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:52.076299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:52.091182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ddbf-8acd-4c41-a808-8c366c0b77fd admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:52.094852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:52.095450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:52.095798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:52.096034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:52.105521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:52.107825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:52.117833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1137/2293] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:52.404049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:52.405530 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:52.423458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29796c-8dcf-4da4-a8ec-b8ce6bd8c17f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:52.435521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:52.436479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:52.436880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:52.437161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] PATH_INFO: `/v3/projects/74c4e0da86aa42e1b5062dab2a174b18` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:52.438449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c4e0da86aa42e1b5062dab2a174b18)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:52.440414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:52.499935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c4e0da86aa42e1b5062dab2a174b18', 'request_id': 'req-ee964af2-ec1d-443c-9c1e-02b7743a5835'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:52.506237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c4e0da86aa42e1b5062dab2a174b18', 'request_id': 'req-ee964af2-ec1d-443c-9c1e-02b7743a5835'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:52.512763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c4e0da86aa42e1b5062dab2a174b18', 'request_id': 'req-ee964af2-ec1d-443c-9c1e-02b7743a5835'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:52.526986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] The token cache is being invalidate because project 74c4e0da86aa42e1b5062dab2a174b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:12:52.527302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:52.528963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:12:52.530641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1157/2294] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:12:52 2022] DELETE /identity/v3/projects/74c4e0da86aa42e1b5062dab2a174b18 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:12:53.064881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:53.066298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:53.140206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e543-5b92-4f75-a458-cf729becc5a4 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:53.202864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:53.203492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:53.203770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:53.204039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:53.205747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:53.266520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:53.268146 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:53.278681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1138/2295] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:54.479981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:54.481729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:54.542811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee964af2-ec1d-443c-9c1e-02b7743a5835 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:54.547283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a9688-1751-45a8-be83-90bae48246e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:54.548034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9688-1751-45a8-be83-90bae48246e3 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:54.548321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9688-1751-45a8-be83-90bae48246e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:54.548656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9688-1751-45a8-be83-90bae48246e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:54.550487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a15a9688-1751-45a8-be83-90bae48246e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:54.565509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a9688-1751-45a8-be83-90bae48246e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:54.567261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a9688-1751-45a8-be83-90bae48246e3 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:54.581262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1158/2296] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:12:56.225411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:12:56.227057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:56.253415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:12:56.287353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d63c05-69ee-45d3-bf88-22e3b4cfd53d service nova] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:12:56.291358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:12:56.291967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:12:56.292244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:12:56.292512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:12:56.294460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:12:56.354881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:12:56.356384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:12:56.367394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1139/2297] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:12:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:04.574615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:04.608478 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:04.608767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:04.609046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:04.629514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] MFA Rules not processed for user `624a0a90866446439918bd65ce5ddc60`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:04.630754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:04.638720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1159/2298] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:05.397737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:05.410749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7090b-746a-4157-8e68-2ee699c57801 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:05.414467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:05.415304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:05.415685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:05.416064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:05.418181 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:05.429407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:05.430966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:05.440270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1140/2299] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:05.588156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:05.589833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:05.607993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e71ec-0ba1-45b9-bef2-c59a7ada6e75 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:05.613601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:05.614368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:05.614504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:05.614733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:05.616502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:05.678061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:05.679828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:05.689464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1160/2300] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:05.769228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:05.770723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:05.865522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879f485-8791-4636-abba-6a6a10309a79 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:05.870632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:05.871269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:05.872055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:05.872329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:05.874669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:05.886748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:05.889120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:05.900247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1141/2301] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:15.686278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=31 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:15.895886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=76 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:18.842849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:18.844309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:18.858577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f96ac9-8d55-4d23-a69e-1e5450b8d930 service nova] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:18.864199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:18.864843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:18.865305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:18.865578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:18.867347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:18.880112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:18.881707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:18.894705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1161/2302] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:19.530416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:19.539869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb4ef1-67c7-440d-a5f6-0b1fbe30e119 service placement] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:19.543841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90268c-0eef-497a-970e-634a2b7a8a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:19.544784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90268c-0eef-497a-970e-634a2b7a8a56 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:19.545678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90268c-0eef-497a-970e-634a2b7a8a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:19.545961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90268c-0eef-497a-970e-634a2b7a8a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:19.547743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5d90268c-0eef-497a-970e-634a2b7a8a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:19.557895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90268c-0eef-497a-970e-634a2b7a8a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:19.559649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90268c-0eef-497a-970e-634a2b7a8a56 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:19.568403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1142/2303] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:23.029277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:23.030773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:23.043209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815d7f-71cc-49d5-a8c5-986997e2f9b9 service nova] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:23.047256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:23.047868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:23.048141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:23.048410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:23.050112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:23.111923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:23.113379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:23.122597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1162/2304] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:23 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:24.642072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f12de-f28d-42ad-8463-604e85342488 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:24.642596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f12de-f28d-42ad-8463-604e85342488 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:24.642596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f12de-f28d-42ad-8463-604e85342488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:24.663244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f02f12de-f28d-42ad-8463-604e85342488 None None] MFA Rules not processed for user `ac0bc9e010d044a5a2147010cbdb4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:24.664500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f02f12de-f28d-42ad-8463-604e85342488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:24.672608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1143/2305] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:24.986102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:24.995659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de6968-86d7-4d47-9706-3bdac9e76a37 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:24.999988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e73a6-adf4-47ed-8523-ec58d0cc3dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:25.000582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e73a6-adf4-47ed-8523-ec58d0cc3dcc service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:25.000864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e73a6-adf4-47ed-8523-ec58d0cc3dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:25.001138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e73a6-adf4-47ed-8523-ec58d0cc3dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:25.002837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f43e73a6-adf4-47ed-8523-ec58d0cc3dcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:25.012490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e73a6-adf4-47ed-8523-ec58d0cc3dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:25.014034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e73a6-adf4-47ed-8523-ec58d0cc3dcc service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:25.022053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1163/2306] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:27.340465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f12de-f28d-42ad-8463-604e85342488 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:27.353478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f12de-f28d-42ad-8463-604e85342488 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:27.358752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:27.359398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:27.359675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:27.359946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:27.361709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:27.375461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:27.377082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:27.388085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1144/2307] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:32.299818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f331f79-fc5b-477c-abd3-b912a700c958 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:32.300313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f331f79-fc5b-477c-abd3-b912a700c958 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:32.300424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f331f79-fc5b-477c-abd3-b912a700c958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:32.322724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8f331f79-fc5b-477c-abd3-b912a700c958 None None] MFA Rules not processed for user `8f1c0b2189a041fdb8dbe2a0290ffac8`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:32.323983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8f331f79-fc5b-477c-abd3-b912a700c958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:32.394322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1164/2308] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:37.383366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=15 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:37.919610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:37.932033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c06b-f17d-4368-8e2e-a08825ba932d service nova] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:37.938831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:37.939661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:37.939947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:37.940223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:37.942103 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:37.958924 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:37.961077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:37.974541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1145/2309] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:42.384955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=65 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:47.968356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=6 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:49.617769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f331f79-fc5b-477c-abd3-b912a700c958 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:49.628895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f331f79-fc5b-477c-abd3-b912a700c958 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:49.632742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:49.633359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:49.633641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:49.633916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:49.636292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:49.708061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:49.709507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:49.721259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1165/2310] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:54.286557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:54.288046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:54.307801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19377032-a4b7-454d-a3ac-764f44aa874f service nova] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:54.312845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:54.314039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:54.314777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:54.314777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] PATH_INFO: `/v3/users/73c4a7ab7fc7425389843798faa25372` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:54.315927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c4a7ab7fc7425389843798faa25372)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:54.317896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:54.372577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c4a7ab7fc7425389843798faa25372'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.385766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c4a7ab7fc7425389843798faa25372'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.386498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c4a7ab7fc7425389843798faa25372'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.387132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c4a7ab7fc7425389843798faa25372'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.394511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7308161d-84cc-49b8-b488-d3691b59432d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c4a7ab7fc7425389843798faa25372'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.408394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1146/2311] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:13:54 2022] DELETE /identity/v3/users/73c4a7ab7fc7425389843798faa25372 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:54.582230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:54.592402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3458e-fa66-498c-9602-bb579f270a68 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:54.650600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:54.651426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:54.651713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:54.652004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] PATH_INFO: `/v3/projects/b429eb37f4ed408d830ef2c1034e9bdc` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:54.653169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b429eb37f4ed408d830ef2c1034e9bdc)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:54.655151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:54.701338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b429eb37f4ed408d830ef2c1034e9bdc', 'request_id': 'req-57d42c3b-771a-41de-ae5f-86def91d8ff1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.708016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b429eb37f4ed408d830ef2c1034e9bdc', 'request_id': 'req-57d42c3b-771a-41de-ae5f-86def91d8ff1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.713562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b429eb37f4ed408d830ef2c1034e9bdc', 'request_id': 'req-57d42c3b-771a-41de-ae5f-86def91d8ff1'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.726199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] The token cache is being invalidate because project b429eb37f4ed408d830ef2c1034e9bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:13:54.726491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.727133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-57d42c3b-771a-41de-ae5f-86def91d8ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:54.728631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1166/2312] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:13:54 2022] DELETE /identity/v3/projects/b429eb37f4ed408d830ef2c1034e9bdc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:54.734211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ae584-0e09-40c6-8696-e9d6fb1dc6a4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:54.734211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ae584-0e09-40c6-8696-e9d6fb1dc6a4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:54.734211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ae584-0e09-40c6-8696-e9d6fb1dc6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:54.755469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-884ae584-0e09-40c6-8696-e9d6fb1dc6a4 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:54.756770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-884ae584-0e09-40c6-8696-e9d6fb1dc6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:54.848233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1147/2313] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:54.855631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faddb5-f66a-485d-adae-a18318e91164 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:54.856156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faddb5-f66a-485d-adae-a18318e91164 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:54.856440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faddb5-f66a-485d-adae-a18318e91164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:54.878022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-31faddb5-f66a-485d-adae-a18318e91164 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:54.879285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-31faddb5-f66a-485d-adae-a18318e91164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:54.887758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1167/2314] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:54.892576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ae584-0e09-40c6-8696-e9d6fb1dc6a4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:54.902315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ae584-0e09-40c6-8696-e9d6fb1dc6a4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:54.907123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:54.908248 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:54.908518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:54.908787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:54.909434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:13:54.909731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:54.911349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:54.926528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1148/2315] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:13:54.929740 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faddb5-f66a-485d-adae-a18318e91164 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:54.941329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faddb5-f66a-485d-adae-a18318e91164 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:54.946543 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:54.947185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:54.947467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:54.947767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:54.948407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:54.950319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:54.968786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:54.991996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1168/2316] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:13:54 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:13:54.993073 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.003962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec5ad3-cfaf-40f8-ae90-ce76cba155b7 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.060724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.061335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.061618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.061895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.062528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.065035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.073588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:55.088778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1149/2317] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:13:54 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:13:55.092927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.103338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cec9e-e700-4296-a94c-0f4fa639371e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.107979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.108571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.108852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.109126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.109764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.111622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.122007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1169/2318] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:13:55.126430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.136388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce14cb-39c3-496c-850e-3ac9fccf01c3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.141902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d77e09-f133-414f-a09b-bb023c57cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.142631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d77e09-f133-414f-a09b-bb023c57cc85 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.142914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d77e09-f133-414f-a09b-bb023c57cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.143225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d77e09-f133-414f-a09b-bb023c57cc85 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.143899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d77e09-f133-414f-a09b-bb023c57cc85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.145482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d77e09-f133-414f-a09b-bb023c57cc85 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.156237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1150/2319] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:13:55.160923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.171179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafadff3-f30e-453c-bcdb-4c668657426d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.175896 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.176540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.176817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.177090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] PATH_INFO: `/v3/projects/e7f21df2e95c4c07b1539f1ec22a4cef/users/1bda07a9b7e14e93a17891a060f43d74/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.192625 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f21df2e95c4c07b1539f1ec22a4cef, user_id=1bda07a9b7e14e93a17891a060f43d74, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.195326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.208686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1170/2320] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:13:55 2022] PUT /identity/v3/projects/e7f21df2e95c4c07b1539f1ec22a4cef/users/1bda07a9b7e14e93a17891a060f43d74/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:55.220059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79f6c5-55e8-44f1-a56e-ff518d4cd544 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.220324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79f6c5-55e8-44f1-a56e-ff518d4cd544 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.220606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79f6c5-55e8-44f1-a56e-ff518d4cd544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.250202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5c79f6c5-55e8-44f1-a56e-ff518d4cd544 None None] MFA Rules not processed for user `1bda07a9b7e14e93a17891a060f43d74`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:55.251959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5c79f6c5-55e8-44f1-a56e-ff518d4cd544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:55.317557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1151/2321] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:55.322051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.384528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bfbfb-64bc-4dcd-8e61-b8b8e989157b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.389173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.389772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.390055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.390336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.390961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.392934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.419322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1171/2322] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:13:55 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:13:55.423984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79f6c5-55e8-44f1-a56e-ff518d4cd544 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.434949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79f6c5-55e8-44f1-a56e-ff518d4cd544 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.495900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.496423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.496706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.496976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.497612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.499553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.523304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1152/2323] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:13:55 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:13:55.527289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.538569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeda81-0c05-48d9-8efb-2aabdcbba7b7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.543222 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.544097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.544404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.544749 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.545383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.546985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.558303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1172/2324] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:13:55.563160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.574321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb67900-0950-4e9d-9861-6f57774cc62c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.579084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.579836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.580240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.580523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] PATH_INFO: `/v3/projects/239f87be45cd498f96ef9ef34463fc3a/users/0eb3cb047fbd490ab66fa016b7da09ff/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.595958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=239f87be45cd498f96ef9ef34463fc3a, user_id=0eb3cb047fbd490ab66fa016b7da09ff, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.598676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.614428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1153/2325] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:13:55 2022] PUT /identity/v3/projects/239f87be45cd498f96ef9ef34463fc3a/users/0eb3cb047fbd490ab66fa016b7da09ff/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:55.618474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.631030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ef69e-c97c-452a-b72f-0667bfaa017b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.691056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc7ea0-1b86-41e5-90bb-cd22a6437f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.691686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7ea0-1b86-41e5-90bb-cd22a6437f00 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.691964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7ea0-1b86-41e5-90bb-cd22a6437f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.692233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7ea0-1b86-41e5-90bb-cd22a6437f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.692862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc7ea0-1b86-41e5-90bb-cd22a6437f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.694475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc7ea0-1b86-41e5-90bb-cd22a6437f00 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.704697 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1173/2326] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:13:55.708918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.720230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eef4e2-19a0-4906-b8a9-3b82b999d4e7 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.724844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.725487 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.725768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.726041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] PATH_INFO: `/v3/projects/239f87be45cd498f96ef9ef34463fc3a/users/0eb3cb047fbd490ab66fa016b7da09ff/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.727975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=239f87be45cd498f96ef9ef34463fc3a, user_id=0eb3cb047fbd490ab66fa016b7da09ff, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.730616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.741398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1154/2327] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:13:55 2022] PUT /identity/v3/projects/239f87be45cd498f96ef9ef34463fc3a/users/0eb3cb047fbd490ab66fa016b7da09ff/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:55.752367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.752661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.753059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.781332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] MFA Rules not processed for user `0eb3cb047fbd490ab66fa016b7da09ff`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:55.782738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:55.801316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:55.853314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1174/2328] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:55.865650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.867427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:55.889784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:55.926837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c8eb4-79df-4198-8fe3-6abeb2c6adf5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:55.930482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:55.931090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:55.931385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:55.931655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:55.941828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:55.943272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:55.952577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1155/2329] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:55.998276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:55.999831 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:56.013499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cd073-455f-4e9a-9fa1-78968edb145d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:56.017966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d6db2-c651-482f-969d-24502a73d908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:56.018584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6db2-c651-482f-969d-24502a73d908 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:56.018868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6db2-c651-482f-969d-24502a73d908 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:56.019181 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6db2-c651-482f-969d-24502a73d908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:56.087048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6db2-c651-482f-969d-24502a73d908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:56.089218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6db2-c651-482f-969d-24502a73d908 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:56.099431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1175/2330] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:13:59.434429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:59.497461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afb294-3171-4d88-a118-afe1d65bb777 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:59.502054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:59.502681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:59.502958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:59.503264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] PATH_INFO: `/v3/users/1bda07a9b7e14e93a17891a060f43d74` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:59.504381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bda07a9b7e14e93a17891a060f43d74)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:59.506299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:59.552890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bda07a9b7e14e93a17891a060f43d74'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:59.565751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bda07a9b7e14e93a17891a060f43d74'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:59.566767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bda07a9b7e14e93a17891a060f43d74'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:59.567060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bda07a9b7e14e93a17891a060f43d74'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:59.574165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bda07a9b7e14e93a17891a060f43d74'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:13:59.589980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1156/2331] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:13:59 2022] DELETE /identity/v3/users/1bda07a9b7e14e93a17891a060f43d74 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:13:59.594634 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:59.595254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:59.595363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:59.616058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:13:59.617301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:59.680445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1176/2332] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:13:59.692632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:13:59.694052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:13:59.741957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:13:59.756250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e48b-60d2-4df8-b685-9d1f6401283d admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:13:59.759939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:13:59.760552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:13:59.760838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:13:59.761107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:13:59.771367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:13:59.772834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:13:59.782980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1157/2333] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:00.212606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:00.214276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:00.227997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6479c7-c3e9-42f2-bd13-c32c0a9047b0 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:00.232811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:00.233513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:00.233793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:00.234133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] PATH_INFO: `/v3/projects/e7f21df2e95c4c07b1539f1ec22a4cef` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:00.235461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f21df2e95c4c07b1539f1ec22a4cef)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:00.238023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:00.288327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f21df2e95c4c07b1539f1ec22a4cef', 'request_id': 'req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.289399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:00.296518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f21df2e95c4c07b1539f1ec22a4cef', 'request_id': 'req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.303162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f21df2e95c4c07b1539f1ec22a4cef', 'request_id': 'req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.315421 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] The token cache is being invalidate because project e7f21df2e95c4c07b1539f1ec22a4cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:00.315713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.316352 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.317832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1177/2334] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:00 2022] DELETE /identity/v3/projects/e7f21df2e95c4c07b1539f1ec22a4cef => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:00.321972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:00.331976 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e66f-8de3-42b9-9abd-e147e1e5b5d4 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:00.389120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:00.389676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:00.389953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:00.390227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] PATH_INFO: `/v3/users/0eb3cb047fbd490ab66fa016b7da09ff` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:00.391445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb3cb047fbd490ab66fa016b7da09ff)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:00.393372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:00.441655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb3cb047fbd490ab66fa016b7da09ff'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.453742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb3cb047fbd490ab66fa016b7da09ff'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.454495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb3cb047fbd490ab66fa016b7da09ff'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.455142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb3cb047fbd490ab66fa016b7da09ff'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.461996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8743d71b-7952-446b-bb50-568ad140bcba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb3cb047fbd490ab66fa016b7da09ff'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.476730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1158/2335] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:00 2022] DELETE /identity/v3/users/0eb3cb047fbd490ab66fa016b7da09ff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:00.758280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:00.759786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:00.774008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eb6fe-94f5-4060-b3d9-0bda6c5d18a5 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:00.837123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:00.837593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:00.837874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:00.838147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] PATH_INFO: `/v3/projects/239f87be45cd498f96ef9ef34463fc3a` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:00.839287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=239f87be45cd498f96ef9ef34463fc3a)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:00.841259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:00.890135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239f87be45cd498f96ef9ef34463fc3a', 'request_id': 'req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.897345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239f87be45cd498f96ef9ef34463fc3a', 'request_id': 'req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.902882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239f87be45cd498f96ef9ef34463fc3a', 'request_id': 'req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.916673 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] The token cache is being invalidate because project 239f87be45cd498f96ef9ef34463fc3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:00.917381 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.917580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-eae0ed01-6d27-41f4-b2cc-6dd4b32f4337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:00.919347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1178/2336] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:00 2022] DELETE /identity/v3/projects/239f87be45cd498f96ef9ef34463fc3a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:00.924873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9fb7-2554-4b28-b0e0-f15baf3494d5 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:00.925164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9fb7-2554-4b28-b0e0-f15baf3494d5 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:00.925449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9fb7-2554-4b28-b0e0-f15baf3494d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:00.945847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5d1e9fb7-2554-4b28-b0e0-f15baf3494d5 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:00.947052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5d1e9fb7-2554-4b28-b0e0-f15baf3494d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:01.010552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1159/2337] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:01.019580 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4e16f-b7b1-41f4-bdd9-b4e7d3847223 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.019856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4e16f-b7b1-41f4-bdd9-b4e7d3847223 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.020134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4e16f-b7b1-41f4-bdd9-b4e7d3847223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.039598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-6cb4e16f-b7b1-41f4-bdd9-b4e7d3847223 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:01.040811 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6cb4e16f-b7b1-41f4-bdd9-b4e7d3847223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:01.049262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1179/2338] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:01.053527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9fb7-2554-4b28-b0e0-f15baf3494d5 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.059631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:01.064050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9fb7-2554-4b28-b0e0-f15baf3494d5 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.070205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.070876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.071365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.071841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.072577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:01.073207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.075048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.087938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1160/2339] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:01.092736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4e16f-b7b1-41f4-bdd9-b4e7d3847223 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.103524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4e16f-b7b1-41f4-bdd9-b4e7d3847223 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.109121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.109734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.110106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.110378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.111024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.112964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.146079 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1180/2340] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:01.150477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.161455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff2aa0-c288-4387-b2a8-2dbb96382874 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.235799 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.236461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.236743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.237022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.237691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.239669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.268069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1161/2341] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:01.269529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.283300 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a6e28-3c82-4172-a68e-b2f6f7a630fd admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.288836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.289691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.289819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.290136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.290882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.292684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.304758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1181/2342] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:01.309157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.320907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be78f8-0609-41ad-b89f-ff5e7ccfb378 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.327824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575a0aa-045f-4f8c-a7d3-08f4d039bb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.328449 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a0aa-045f-4f8c-a7d3-08f4d039bb58 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.328738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a0aa-045f-4f8c-a7d3-08f4d039bb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.329109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a0aa-045f-4f8c-a7d3-08f4d039bb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.329860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a0aa-045f-4f8c-a7d3-08f4d039bb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.331589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a0aa-045f-4f8c-a7d3-08f4d039bb58 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.343813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1162/2343] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:01.347949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.359431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dbaf7-8a28-4f41-8273-ebab44f57ade admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.365539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.366368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.366659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.367563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] PATH_INFO: `/v3/projects/e864f7f9389b4da2847fe1e7901decaf/users/2637771425414faf89898426bffa9422/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.389305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e864f7f9389b4da2847fe1e7901decaf, user_id=2637771425414faf89898426bffa9422, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.393584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.415356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1182/2344] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:01 2022] PUT /identity/v3/projects/e864f7f9389b4da2847fe1e7901decaf/users/2637771425414faf89898426bffa9422/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:01.428693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880becc-c558-4e9f-a7cb-b615c44e0217 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.428982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880becc-c558-4e9f-a7cb-b615c44e0217 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.429261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880becc-c558-4e9f-a7cb-b615c44e0217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.458434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8880becc-c558-4e9f-a7cb-b615c44e0217 None None] MFA Rules not processed for user `2637771425414faf89898426bffa9422`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:01.459880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8880becc-c558-4e9f-a7cb-b615c44e0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:01.529542 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1163/2345] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:01.534132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.539787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:01.599416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7f9f-af80-4ac5-a2c5-83e9444b5ef5 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.604129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.604739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.605024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.605303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.605950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.607909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.638223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1183/2346] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:01.642090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880becc-c558-4e9f-a7cb-b615c44e0217 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.653836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880becc-c558-4e9f-a7cb-b615c44e0217 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.720633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.721250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.721538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.721876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.722560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.724845 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.734230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:01.753441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1164/2347] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:01.754670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.766173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d82507-49c2-4117-b2a5-f6b066d0329f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.771817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.772650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.772773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.773008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.773655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.775288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.787746 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1184/2348] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:01.791089 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.802624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005db6-550f-4eeb-97b3-ce2d25d712a4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.807344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.808009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.808558 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.808838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] PATH_INFO: `/v3/projects/892491a1554048cf8f8c7293a2f7f06c/users/e6aba9dd4838441baa559df36dff8b30/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.825610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892491a1554048cf8f8c7293a2f7f06c, user_id=e6aba9dd4838441baa559df36dff8b30, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.829786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.844904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1165/2349] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:01 2022] PUT /identity/v3/projects/892491a1554048cf8f8c7293a2f7f06c/users/e6aba9dd4838441baa559df36dff8b30/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:01.849213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.860414 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e314-623b-45bf-9e61-985f1996de13 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.923943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c119c-c676-469c-a3e3-7e2687d3d73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.924474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c119c-c676-469c-a3e3-7e2687d3d73e admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.924753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c119c-c676-469c-a3e3-7e2687d3d73e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.925023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c119c-c676-469c-a3e3-7e2687d3d73e admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.925665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c119c-c676-469c-a3e3-7e2687d3d73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.927299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c119c-c676-469c-a3e3-7e2687d3d73e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.938706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1185/2350] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:01.942797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:01.954591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017faf1-a096-4ed9-ba8e-ef760b8ed872 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:01.959432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:01.960148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.960428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.960699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] PATH_INFO: `/v3/projects/892491a1554048cf8f8c7293a2f7f06c/users/e6aba9dd4838441baa559df36dff8b30/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:01.962645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=892491a1554048cf8f8c7293a2f7f06c, user_id=e6aba9dd4838441baa559df36dff8b30, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:01.965285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:01.977590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1166/2351] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:01 2022] PUT /identity/v3/projects/892491a1554048cf8f8c7293a2f7f06c/users/e6aba9dd4838441baa559df36dff8b30/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:01.988575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4f43-10eb-4604-bc3f-589b073c3dfc None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:01.988864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4f43-10eb-4604-bc3f-589b073c3dfc None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:01.989151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f4f43-10eb-4604-bc3f-589b073c3dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.017615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-641f4f43-10eb-4604-bc3f-589b073c3dfc None None] MFA Rules not processed for user `e6aba9dd4838441baa559df36dff8b30`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:02.018834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-641f4f43-10eb-4604-bc3f-589b073c3dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:02.087193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:02.091127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1186/2352] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:02.104415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:02.106108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:02.164195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36efcd-f8d1-4a7c-b267-93c979560d2b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:02.167932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:02.168530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.168802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.169067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.178410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:02.179785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:02.189299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1167/2353] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:02.537875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f4f43-10eb-4604-bc3f-589b073c3dfc None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:02.548652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f4f43-10eb-4604-bc3f-589b073c3dfc None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:02.553716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6aba9dd4838441baa559df36dff8b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '892491a1554048cf8f8c7293a2f7f06c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:02.554405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.554755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.555189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.555914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:02.556291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:02.557999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:02.570592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1187/2354] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:14:02.575163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:02.585175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88636a-84d7-46cb-b418-931f58d04207 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:02.589858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6aba9dd4838441baa559df36dff8b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '892491a1554048cf8f8c7293a2f7f06c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:02.590780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.590780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.591059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.591754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:02.593965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:02.630186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1168/2355] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:02 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:02.635139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:02.645884 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857b4c-6b4a-4a9e-92ae-af144cd2cd6a tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:02.711227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af352e-2696-4675-9f4d-a6811b1937e7 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6aba9dd4838441baa559df36dff8b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '892491a1554048cf8f8c7293a2f7f06c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:02.711849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af352e-2696-4675-9f4d-a6811b1937e7 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.712137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af352e-2696-4675-9f4d-a6811b1937e7 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.712415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af352e-2696-4675-9f4d-a6811b1937e7 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.713097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af352e-2696-4675-9f4d-a6811b1937e7 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:02.713437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af352e-2696-4675-9f4d-a6811b1937e7 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:02.715190 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af352e-2696-4675-9f4d-a6811b1937e7 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:02.728705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1188/2356] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:02.767488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:02.777812 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec416a2-3149-481a-bc48-02b7de4b5786 tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:02.783451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6aba9dd4838441baa559df36dff8b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '892491a1554048cf8f8c7293a2f7f06c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:02.784104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.784379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.784647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] PATH_INFO: `/v3/projects/aad2c17dd9e644f48e07becc2f5b7419` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.785831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aad2c17dd9e644f48e07becc2f5b7419)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:02.787788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:02.847459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad2c17dd9e644f48e07becc2f5b7419', 'request_id': 'req-5bc15232-d03a-4f60-870d-5109e4f24d0d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:02.856115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad2c17dd9e644f48e07becc2f5b7419', 'request_id': 'req-5bc15232-d03a-4f60-870d-5109e4f24d0d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:02.864131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad2c17dd9e644f48e07becc2f5b7419', 'request_id': 'req-5bc15232-d03a-4f60-870d-5109e4f24d0d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:02.873513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066af69-b8b2-4b78-93a1-02544d07ef92 None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.874039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066af69-b8b2-4b78-93a1-02544d07ef92 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.874039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066af69-b8b2-4b78-93a1-02544d07ef92 None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.875452 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1189/2357] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:14:02.875826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] The token cache is being invalidate because project aad2c17dd9e644f48e07becc2f5b7419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:02.876391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:02.877154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:02.879016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1169/2358] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:02 2022] DELETE /identity/v3/projects/aad2c17dd9e644f48e07becc2f5b7419 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:02.886650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:02.887230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:02.887586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:02.912166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] MFA Rules not processed for user `70c6682b92694484b2fa2710314f547f`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:02.913506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:02.977279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1190/2359] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:14:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:02.995577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:02.997023 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:03.010821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc15232-d03a-4f60-870d-5109e4f24d0d tempest-VolumeQuotaClassesTest-1709506427 tempest-VolumeQuotaClassesTest-1709506427-project-admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:03.014738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac3100-8dc8-47f4-a188-8424da85ce16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:03.015379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac3100-8dc8-47f4-a188-8424da85ce16 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.015658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac3100-8dc8-47f4-a188-8424da85ce16 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.015931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac3100-8dc8-47f4-a188-8424da85ce16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.038507 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac3100-8dc8-47f4-a188-8424da85ce16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:03.040058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac3100-8dc8-47f4-a188-8424da85ce16 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:03.048525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1170/2360] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:03.099801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:03.101249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:03.140483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f292a0-09a6-4e7f-913b-b178e88adee7 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.140847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f292a0-09a6-4e7f-913b-b178e88adee7 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.141193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f292a0-09a6-4e7f-913b-b178e88adee7 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.143567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1171/2361] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:14:03.152972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c849e-ab4c-491f-9a20-7b0ad06e2ba8 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.152972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c849e-ab4c-491f-9a20-7b0ad06e2ba8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.152972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c849e-ab4c-491f-9a20-7b0ad06e2ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.159249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:INSERT=7,keystone:SELECT=86,keystone:DELETE=7 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:03.171575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fc0f8-e06b-4389-a1a8-3e10c68687bc None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:03.173785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bc1c849e-ab4c-491f-9a20-7b0ad06e2ba8 None None] MFA Rules not processed for user `8adead3860f144099efa11cefa83a37f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:03.175333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bc1c849e-ab4c-491f-9a20-7b0ad06e2ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:03.176730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:03.177366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.177645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.177918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] PATH_INFO: `/v3/users/2637771425414faf89898426bffa9422` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.179088 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2637771425414faf89898426bffa9422)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:03.181046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:03.219837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:03.239575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2637771425414faf89898426bffa9422'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.254774 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1172/2362] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:14:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:03.258269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2637771425414faf89898426bffa9422'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.259104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2637771425414faf89898426bffa9422'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.265539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c849e-ab4c-491f-9a20-7b0ad06e2ba8 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:03.274729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2637771425414faf89898426bffa9422'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.276074 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c849e-ab4c-491f-9a20-7b0ad06e2ba8 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:03.280548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:03.281272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.281510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.281873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.284356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dedd750-39ba-419f-aca5-7c94b311e01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2637771425414faf89898426bffa9422'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.293939 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1191/2363] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:03 2022] DELETE /identity/v3/users/2637771425414faf89898426bffa9422 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:03.295705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:03.296023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.296314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.296781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.297438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:03.311454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1173/2364] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:14:03.317233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:03.318472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:03.388256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1192/2365] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:03.399015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:03.408877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ada0af-7170-4e40-9f89-f26cdef6533a service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:03.472895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:03.473518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.473819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.474062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.484131 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:03.495411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:03.498619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1174/2366] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:03.820579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:03.822211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:03.838161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287d17d-e825-47dc-b3ed-3b87f0c97687 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:03.843847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:03.845179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:03.846989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:03.846989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] PATH_INFO: `/v3/projects/e864f7f9389b4da2847fe1e7901decaf` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:03.847218 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e864f7f9389b4da2847fe1e7901decaf)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:03.849247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:03.922475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e864f7f9389b4da2847fe1e7901decaf', 'request_id': 'req-7da8c4a4-1045-4abd-95d7-a76e11f80250'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.929154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e864f7f9389b4da2847fe1e7901decaf', 'request_id': 'req-7da8c4a4-1045-4abd-95d7-a76e11f80250'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.935158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e864f7f9389b4da2847fe1e7901decaf', 'request_id': 'req-7da8c4a4-1045-4abd-95d7-a76e11f80250'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.949448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] The token cache is being invalidate because project e864f7f9389b4da2847fe1e7901decaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:03.949743 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.950368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:03.951880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1193/2367] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:03 2022] DELETE /identity/v3/projects/e864f7f9389b4da2847fe1e7901decaf => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:03.956849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:03.966530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65889e-20ab-4cf3-a148-5018639310c1 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:04.024277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:04.024995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.025277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.025547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] PATH_INFO: `/v3/users/e6aba9dd4838441baa559df36dff8b30` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.026669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6aba9dd4838441baa559df36dff8b30)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:04.028707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:04.060902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:04.085054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6aba9dd4838441baa559df36dff8b30'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.097602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6aba9dd4838441baa559df36dff8b30'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.098303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6aba9dd4838441baa559df36dff8b30'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.098904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6aba9dd4838441baa559df36dff8b30'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.106379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-217466bc-c1a4-4248-a23e-219b476ac113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6aba9dd4838441baa559df36dff8b30'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.120280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1175/2368] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:03 2022] DELETE /identity/v3/users/e6aba9dd4838441baa559df36dff8b30 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:04.399358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:04.400877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:04.415202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8c4a4-1045-4abd-95d7-a76e11f80250 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:04.447263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:04.476730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:04.477313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.477594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.477867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] PATH_INFO: `/v3/projects/892491a1554048cf8f8c7293a2f7f06c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.479407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] RBAC: Authorizing `identity:delete_project(project_id=892491a1554048cf8f8c7293a2f7f06c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:04.481507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:04.528949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892491a1554048cf8f8c7293a2f7f06c', 'request_id': 'req-6cfcc192-083d-471c-b942-cc6dfbf722be'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.536547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892491a1554048cf8f8c7293a2f7f06c', 'request_id': 'req-6cfcc192-083d-471c-b942-cc6dfbf722be'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.542273 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892491a1554048cf8f8c7293a2f7f06c', 'request_id': 'req-6cfcc192-083d-471c-b942-cc6dfbf722be'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.556872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] The token cache is being invalidate because project 892491a1554048cf8f8c7293a2f7f06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:04.557165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.557808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cfcc192-083d-471c-b942-cc6dfbf722be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:04.559531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1194/2369] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:04 2022] DELETE /identity/v3/projects/892491a1554048cf8f8c7293a2f7f06c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:04.564888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e5773-87a2-4fa8-b800-72bc3a378189 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.565176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e5773-87a2-4fa8-b800-72bc3a378189 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.565457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e5773-87a2-4fa8-b800-72bc3a378189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.587031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-dd9e5773-87a2-4fa8-b800-72bc3a378189 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:04.588290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-dd9e5773-87a2-4fa8-b800-72bc3a378189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:04.661054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1176/2370] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:04.670089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5291d-4f62-4538-938d-e91311353167 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.670373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5291d-4f62-4538-938d-e91311353167 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.670649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5291d-4f62-4538-938d-e91311353167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.693985 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-34c5291d-4f62-4538-938d-e91311353167 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:04.695731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-34c5291d-4f62-4538-938d-e91311353167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:04.705388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1195/2371] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:04.709835 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e5773-87a2-4fa8-b800-72bc3a378189 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:04.721762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e5773-87a2-4fa8-b800-72bc3a378189 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:04.726879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:04.727942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.728281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.728412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.729064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:04.729498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:04.731125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:04.744848 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1177/2372] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:04.749539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5291d-4f62-4538-938d-e91311353167 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:04.760288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5291d-4f62-4538-938d-e91311353167 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:04.765016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:04.765653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.765937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.766231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.767500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:04.769443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:04.797128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1196/2373] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:04 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:04.801717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:04.812383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc31f8-89c1-45f6-a9a3-43e4f1f872e6 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:04.885709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:04.886713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.887206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.887468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.888348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:04.890497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:04.919918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1178/2374] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:04 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:04.925930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:04.953450 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6958e-30f2-4e82-9dfc-4d3f53aeab50 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:04.960394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:04.961022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:04.961307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:04.961579 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:04.962235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:04.964055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:04.978304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1197/2375] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:04.982751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:04.996781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7229f-a2c3-4483-b3c8-4d4ad683625f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.006405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95f539-deba-476d-a56c-303118474367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.007045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95f539-deba-476d-a56c-303118474367 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.010130 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95f539-deba-476d-a56c-303118474367 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.010545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95f539-deba-476d-a56c-303118474367 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.011277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95f539-deba-476d-a56c-303118474367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.014632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95f539-deba-476d-a56c-303118474367 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.031622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1179/2376] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:05.036858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.049465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17d735-bfee-4ca4-bf82-46b7f4a06729 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.055104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.055889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.056275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.056597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] PATH_INFO: `/v3/projects/6bd985248fee469db78452c212e9ff19/users/7b92788f820a4f9287b5333c546aecc1/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.082482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd985248fee469db78452c212e9ff19, user_id=7b92788f820a4f9287b5333c546aecc1, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.086751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.105936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1198/2377] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:05 2022] PUT /identity/v3/projects/6bd985248fee469db78452c212e9ff19/users/7b92788f820a4f9287b5333c546aecc1/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:05.116978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd16f7e-487b-4f77-baab-808bf4f87f5f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.116978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd16f7e-487b-4f77-baab-808bf4f87f5f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.117136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd16f7e-487b-4f77-baab-808bf4f87f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.140929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:05.150749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0bd16f7e-487b-4f77-baab-808bf4f87f5f None None] MFA Rules not processed for user `7b92788f820a4f9287b5333c546aecc1`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:05.152685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0bd16f7e-487b-4f77-baab-808bf4f87f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:05.230743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1180/2378] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:05.235373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.299859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43810e-2bd9-46f4-806d-b1b620ac48b8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.304498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.305105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.305385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.305658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.306309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.308394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.313778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:05.341852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1199/2379] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:05 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:05.343022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd16f7e-487b-4f77-baab-808bf4f87f5f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.354257 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd16f7e-487b-4f77-baab-808bf4f87f5f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.412771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.413391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.413671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.413941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.414583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.416622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.437815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1181/2380] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:05 2022] POST /identity/v3/users => generated 306 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:05.442460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.453227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b7d89-c36a-42f2-8f91-a18320ba0f59 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.458911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.459566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.459841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.460112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.460950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.462531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.473798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1200/2381] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:05.477955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.489237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd325e0-24d0-42d8-ba75-fdeea73ce3c9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.493877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.494524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.494807 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.495117 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] PATH_INFO: `/v3/projects/d277c337cd9d4c4a8839a497ed9a0b14/users/3379e34ebe704f3aa7a772c53c007f97/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.510625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d277c337cd9d4c4a8839a497ed9a0b14, user_id=3379e34ebe704f3aa7a772c53c007f97, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.513382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.527268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1182/2382] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:05 2022] PUT /identity/v3/projects/d277c337cd9d4c4a8839a497ed9a0b14/users/3379e34ebe704f3aa7a772c53c007f97/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:05.531425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.542583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a0c94-7fae-4908-9327-bc0859d19193 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.604651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b50f7-11d4-442c-bd6a-aae93c3aa981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.605472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b50f7-11d4-442c-bd6a-aae93c3aa981 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.605925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b50f7-11d4-442c-bd6a-aae93c3aa981 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.606294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b50f7-11d4-442c-bd6a-aae93c3aa981 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.607044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b50f7-11d4-442c-bd6a-aae93c3aa981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.608977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b50f7-11d4-442c-bd6a-aae93c3aa981 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.624130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1201/2383] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:05.626543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.638469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eed22-37ec-41b7-91d7-69c0738450b6 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.643516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.644174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.644491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.644762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] PATH_INFO: `/v3/projects/d277c337cd9d4c4a8839a497ed9a0b14/users/3379e34ebe704f3aa7a772c53c007f97/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.646679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d277c337cd9d4c4a8839a497ed9a0b14, user_id=3379e34ebe704f3aa7a772c53c007f97, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.649359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.660830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1183/2384] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:05 2022] PUT /identity/v3/projects/d277c337cd9d4c4a8839a497ed9a0b14/users/3379e34ebe704f3aa7a772c53c007f97/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:05.671481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921233-6427-4210-98bb-0a951e78c0a9 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.671770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921233-6427-4210-98bb-0a951e78c0a9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.672044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921233-6427-4210-98bb-0a951e78c0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.702476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-bd921233-6427-4210-98bb-0a951e78c0a9 None None] MFA Rules not processed for user `3379e34ebe704f3aa7a772c53c007f97`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:05.703824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bd921233-6427-4210-98bb-0a951e78c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:05.775587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1202/2385] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:05.786567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.788319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:05.855199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df57527-dd73-413a-ad8f-0f5bf54eef6d admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.858755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.859534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.859809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.860597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:05.866701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:05.870912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:05.873437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:05.884656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1184/2386] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:05.968536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921233-6427-4210-98bb-0a951e78c0a9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:05.979854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921233-6427-4210-98bb-0a951e78c0a9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:05.983751 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01370fc-bb69-459e-b3ab-83f14e6bd4d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:05.984533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01370fc-bb69-459e-b3ab-83f14e6bd4d8 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:05.984921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01370fc-bb69-459e-b3ab-83f14e6bd4d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:05.986060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01370fc-bb69-459e-b3ab-83f14e6bd4d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:06.054411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:06.058843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01370fc-bb69-459e-b3ab-83f14e6bd4d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:06.060792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01370fc-bb69-459e-b3ab-83f14e6bd4d8 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:06.092053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1203/2387] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:05 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:10.234851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:10.299210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68c8cb-5a3b-4f41-aec0-af0e9dce5d00 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:10.303653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:10.304312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:10.304591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:10.304864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] PATH_INFO: `/v3/users/7b92788f820a4f9287b5333c546aecc1` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:10.305984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b92788f820a4f9287b5333c546aecc1)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:10.307984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:10.354357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b92788f820a4f9287b5333c546aecc1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.367099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b92788f820a4f9287b5333c546aecc1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.367822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b92788f820a4f9287b5333c546aecc1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.368465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b92788f820a4f9287b5333c546aecc1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.375685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b92788f820a4f9287b5333c546aecc1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.389918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1185/2388] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:10 2022] DELETE /identity/v3/users/7b92788f820a4f9287b5333c546aecc1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:10.395386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:10.395841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:10.395937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:10.415505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:10.416705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:10.476810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1204/2389] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:10.486946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:10.488415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:10.549415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfafd3-ffda-4925-8173-c44e8f10a081 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:10.553009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:10.553618 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:10.553898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:10.554169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:10.563017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:10.564564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:10.573851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1186/2390] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:10.846763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:10.848297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:10.861324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530eefd-ca5f-4156-af92-87380ffced76 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:10.866950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:10.867635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:10.867917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:10.868189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] PATH_INFO: `/v3/projects/6bd985248fee469db78452c212e9ff19` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:10.869316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd985248fee469db78452c212e9ff19)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:10.871271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:10.918384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd985248fee469db78452c212e9ff19', 'request_id': 'req-50ece302-75fd-4530-8f77-fcfa62775fbd'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.925130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd985248fee469db78452c212e9ff19', 'request_id': 'req-50ece302-75fd-4530-8f77-fcfa62775fbd'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.932286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd985248fee469db78452c212e9ff19', 'request_id': 'req-50ece302-75fd-4530-8f77-fcfa62775fbd'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.946424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] The token cache is being invalidate because project 6bd985248fee469db78452c212e9ff19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:10.946713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.947395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:10.948962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1205/2391] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:10 2022] DELETE /identity/v3/projects/6bd985248fee469db78452c212e9ff19 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:10.953891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:10.964401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559ee2d-32b9-463b-8268-e2413514a85b service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:10.976302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:11.033989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.034635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.034917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.035222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] PATH_INFO: `/v3/users/3379e34ebe704f3aa7a772c53c007f97` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.036364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3379e34ebe704f3aa7a772c53c007f97)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.038294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:11.091597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3379e34ebe704f3aa7a772c53c007f97'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.105085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3379e34ebe704f3aa7a772c53c007f97'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.105797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3379e34ebe704f3aa7a772c53c007f97'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.106408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3379e34ebe704f3aa7a772c53c007f97'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.115865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-90c39c59-62bc-4f4e-b31a-f419b1e22e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3379e34ebe704f3aa7a772c53c007f97'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.131473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1187/2392] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:10 2022] DELETE /identity/v3/users/3379e34ebe704f3aa7a772c53c007f97 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:11.404101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:11.405560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:11.420922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ece302-75fd-4530-8f77-fcfa62775fbd admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:11.481968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.482600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.482885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.483196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] PATH_INFO: `/v3/projects/d277c337cd9d4c4a8839a497ed9a0b14` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.484318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d277c337cd9d4c4a8839a497ed9a0b14)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.486904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:11.508670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:11.539760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd277c337cd9d4c4a8839a497ed9a0b14', 'request_id': 'req-f872495b-7a97-43f7-bad7-7d92727cc0df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.546801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd277c337cd9d4c4a8839a497ed9a0b14', 'request_id': 'req-f872495b-7a97-43f7-bad7-7d92727cc0df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.554090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd277c337cd9d4c4a8839a497ed9a0b14', 'request_id': 'req-f872495b-7a97-43f7-bad7-7d92727cc0df'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.568729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] The token cache is being invalidate because project d277c337cd9d4c4a8839a497ed9a0b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:11.568977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.569637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f872495b-7a97-43f7-bad7-7d92727cc0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:11.571129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1206/2393] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:11 2022] DELETE /identity/v3/projects/d277c337cd9d4c4a8839a497ed9a0b14 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:11.576489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f328a7-7b07-43c0-87b8-7f31e0d48c32 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.576950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f328a7-7b07-43c0-87b8-7f31e0d48c32 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.577220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f328a7-7b07-43c0-87b8-7f31e0d48c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.598198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d7f328a7-7b07-43c0-87b8-7f31e0d48c32 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:11.599553 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d7f328a7-7b07-43c0-87b8-7f31e0d48c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:11.659851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1188/2394] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:11.668609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dbb4f-6f8d-4c2c-a04a-594215b40851 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.668890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dbb4f-6f8d-4c2c-a04a-594215b40851 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.669165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dbb4f-6f8d-4c2c-a04a-594215b40851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.689826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d73dbb4f-6f8d-4c2c-a04a-594215b40851 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:11.691016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d73dbb4f-6f8d-4c2c-a04a-594215b40851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:11.698779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1207/2395] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:11.702950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f328a7-7b07-43c0-87b8-7f31e0d48c32 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:11.712896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f328a7-7b07-43c0-87b8-7f31e0d48c32 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:11.717476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.718065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.718346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.718616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.719291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:11.720138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.721828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:11.733747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1189/2396] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:11.739363 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dbb4f-6f8d-4c2c-a04a-594215b40851 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:11.748961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dbb4f-6f8d-4c2c-a04a-594215b40851 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:11.754410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.755061 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.755393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.755667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.756299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.758197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:11.787481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1208/2397] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:11 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:11.791759 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:11.803230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4037-6500-4cce-ba8b-f315945858e8 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:11.821882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:11.862761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.863441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.863720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.863986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.864651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.866557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:11.887808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1190/2398] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:11 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:11.892554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:11.902768 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe785d-d0c7-488d-9111-bc64699509f1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:11.907980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.908572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.908857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.909132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.909769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.911389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:11.921485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1209/2399] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:11.926005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:11.935964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43187321-fb62-483c-bb27-ed30fd736b00 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:11.940787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78127bba-78df-4108-aab2-8574a02e49cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.941495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78127bba-78df-4108-aab2-8574a02e49cb admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.941866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78127bba-78df-4108-aab2-8574a02e49cb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.942256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78127bba-78df-4108-aab2-8574a02e49cb admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.943028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78127bba-78df-4108-aab2-8574a02e49cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.944739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78127bba-78df-4108-aab2-8574a02e49cb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:11.955890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1191/2400] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:11.960011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:11.970738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8056755-6d50-4d0a-bbfa-9d6202186576 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:11.975475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:11.976121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:11.976404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:11.976678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] PATH_INFO: `/v3/projects/54bcb15a0ab74511a9fd6918dcab89fb/users/f48e3abb43be4f6894b5d1ad3bb1d363/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:11.991250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] RBAC: Authorizing `identity:create_grant(project_id=54bcb15a0ab74511a9fd6918dcab89fb, user_id=f48e3abb43be4f6894b5d1ad3bb1d363, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:11.993840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:12.006482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1210/2401] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:11 2022] PUT /identity/v3/projects/54bcb15a0ab74511a9fd6918dcab89fb/users/f48e3abb43be4f6894b5d1ad3bb1d363/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:12.017676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd7f8e-f0d0-4979-a46a-4cc924c27181 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.018056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd7f8e-f0d0-4979-a46a-4cc924c27181 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.018245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd7f8e-f0d0-4979-a46a-4cc924c27181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:12.046782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-95bd7f8e-f0d0-4979-a46a-4cc924c27181 None None] MFA Rules not processed for user `f48e3abb43be4f6894b5d1ad3bb1d363`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:12.048536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-95bd7f8e-f0d0-4979-a46a-4cc924c27181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:12.113296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1192/2402] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:12.124607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:12.127015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:12.185934 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31501eb2-ff50-4fb1-b1ab-9970576f907a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:12.190432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc8899-3f2c-48f5-a47f-5c5768493189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:12.191042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc8899-3f2c-48f5-a47f-5c5768493189 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.191349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc8899-3f2c-48f5-a47f-5c5768493189 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.191619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc8899-3f2c-48f5-a47f-5c5768493189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:12.202872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc8899-3f2c-48f5-a47f-5c5768493189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:12.204652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc8899-3f2c-48f5-a47f-5c5768493189 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:12.213899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1211/2403] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:12.237348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd7f8e-f0d0-4979-a46a-4cc924c27181 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:12.303358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd7f8e-f0d0-4979-a46a-4cc924c27181 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:12.307972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:12.308610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.308894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.309173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] PATH_INFO: `/v3/users/f48e3abb43be4f6894b5d1ad3bb1d363` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:12.310544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48e3abb43be4f6894b5d1ad3bb1d363)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:12.323657 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:12.352773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:12.368469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48e3abb43be4f6894b5d1ad3bb1d363'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.382588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48e3abb43be4f6894b5d1ad3bb1d363'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.383370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48e3abb43be4f6894b5d1ad3bb1d363'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.384005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48e3abb43be4f6894b5d1ad3bb1d363'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.391328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48e3abb43be4f6894b5d1ad3bb1d363'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.409024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1193/2404] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:12 2022] DELETE /identity/v3/users/f48e3abb43be4f6894b5d1ad3bb1d363 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:12.413507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.413989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.414083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:12.433626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:12.434938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:12.480231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:12.498570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1212/2405] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:12.509081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:12.510518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:12.570217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8147e-b08f-4439-b619-eaef9d4a6d09 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:12.573882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd79f7-ea54-4b66-b965-d710d7d9f74f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:12.574511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd79f7-ea54-4b66-b965-d710d7d9f74f service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.574790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd79f7-ea54-4b66-b965-d710d7d9f74f service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.575063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd79f7-ea54-4b66-b965-d710d7d9f74f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:12.584050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd79f7-ea54-4b66-b965-d710d7d9f74f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:12.585470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd79f7-ea54-4b66-b965-d710d7d9f74f service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:12.595577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1194/2406] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:12.872075 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:12.873555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:12.887549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d79932-8e04-471a-b885-f58f293d8c0e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:12.892627 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:12.893266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.893626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.893901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] PATH_INFO: `/v3/projects/54bcb15a0ab74511a9fd6918dcab89fb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:12.895153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54bcb15a0ab74511a9fd6918dcab89fb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:12.897254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:12.949174 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54bcb15a0ab74511a9fd6918dcab89fb', 'request_id': 'req-d4aa406a-a235-48bf-bda1-1a81203bace7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.955916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54bcb15a0ab74511a9fd6918dcab89fb', 'request_id': 'req-d4aa406a-a235-48bf-bda1-1a81203bace7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.961529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54bcb15a0ab74511a9fd6918dcab89fb', 'request_id': 'req-d4aa406a-a235-48bf-bda1-1a81203bace7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.974396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] The token cache is being invalidate because project 54bcb15a0ab74511a9fd6918dcab89fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:12.974688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.975328 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4aa406a-a235-48bf-bda1-1a81203bace7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:12.976812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1213/2407] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:12 2022] DELETE /identity/v3/projects/54bcb15a0ab74511a9fd6918dcab89fb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:12.981619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0377d9d-24bf-4a12-9057-4b88fd9a09e3 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:12.981910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0377d9d-24bf-4a12-9057-4b88fd9a09e3 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:12.982188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0377d9d-24bf-4a12-9057-4b88fd9a09e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.002541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-b0377d9d-24bf-4a12-9057-4b88fd9a09e3 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:13.003784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b0377d9d-24bf-4a12-9057-4b88fd9a09e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:13.066287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1195/2408] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:13.080249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fcc96-c404-46cb-ae3e-b297459abcda None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.080534 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fcc96-c404-46cb-ae3e-b297459abcda None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.080808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fcc96-c404-46cb-ae3e-b297459abcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.101073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-516fcc96-c404-46cb-ae3e-b297459abcda None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:13.102266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-516fcc96-c404-46cb-ae3e-b297459abcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:13.112227 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1214/2409] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:13.116051 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0377d9d-24bf-4a12-9057-4b88fd9a09e3 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.125686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0377d9d-24bf-4a12-9057-4b88fd9a09e3 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.131527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.132066 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.132338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.132612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.133262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:13.133567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.135204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.147620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1196/2410] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:13.152013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fcc96-c404-46cb-ae3e-b297459abcda None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.161289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fcc96-c404-46cb-ae3e-b297459abcda None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.166508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.167239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.167592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.167938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.168664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.171083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.199922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1215/2411] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:13 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:13.204169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.214985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed5f7-3da1-4fec-87df-59330c71ee79 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.254422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:13.275855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.277122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.277410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.277690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.278875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.281429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.303716 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1197/2412] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:13 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:13.307969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.323739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b890cf-06df-419d-b950-6db82dc910b2 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.328694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.329476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.329796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.330254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.330725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.332374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.347477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.354091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1216/2413] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:13.359314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a053f-7c08-4118-8452-7b8aedc06b52 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.364984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ddfbb-3bd8-4d40-9cd5-39aa3107d358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.365705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ddfbb-3bd8-4d40-9cd5-39aa3107d358 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.366085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ddfbb-3bd8-4d40-9cd5-39aa3107d358 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.366506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ddfbb-3bd8-4d40-9cd5-39aa3107d358 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.367273 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ddfbb-3bd8-4d40-9cd5-39aa3107d358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.369129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ddfbb-3bd8-4d40-9cd5-39aa3107d358 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.389038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.395162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1198/2414] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:14:13.410641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339764-8cb3-4783-8852-991dce1f4fb7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.415510 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.416276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.416628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.416964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] PATH_INFO: `/v3/projects/606b72ef709847d9a8f3a9a822f4eeac/users/e007e9e208ab49aab973901c7398156a/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.439202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] RBAC: Authorizing `identity:create_grant(project_id=606b72ef709847d9a8f3a9a822f4eeac, user_id=e007e9e208ab49aab973901c7398156a, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.442354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.457776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1217/2415] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:13 2022] PUT /identity/v3/projects/606b72ef709847d9a8f3a9a822f4eeac/users/e007e9e208ab49aab973901c7398156a/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:13.469203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad156cf5-57d9-4c67-bfe6-f973bf999491 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.469490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad156cf5-57d9-4c67-bfe6-f973bf999491 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.470404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad156cf5-57d9-4c67-bfe6-f973bf999491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.503121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ad156cf5-57d9-4c67-bfe6-f973bf999491 None None] MFA Rules not processed for user `e007e9e208ab49aab973901c7398156a`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:13.504638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ad156cf5-57d9-4c67-bfe6-f973bf999491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:13.578414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1199/2416] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:13.583857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.672424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6209c4e-caac-4fa2-9a58-fd100925870a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.678171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.678764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.679045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.679346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.679989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.682018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.700865 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:13.731111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1218/2417] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:13 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:13.736442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad156cf5-57d9-4c67-bfe6-f973bf999491 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.748474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad156cf5-57d9-4c67-bfe6-f973bf999491 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.818653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.819312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.819599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.819876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.820530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.822446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.854229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1200/2418] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:13 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:13.856069 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.867350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f869b1a-dc48-4dd9-85e3-58963070d11a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.872253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.872852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.873131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.873401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.874038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.875661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.892056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1219/2419] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:13.893065 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.904356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08a839-624d-4f2c-a11c-c118fb180846 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:13.909651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:13.910309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:13.910589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:13.910883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] PATH_INFO: `/v3/projects/849e4b4603a042a0886b5e4fe2d8cc38/users/95516d15534c4fc3aa6402a4ead06966/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:13.926955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] RBAC: Authorizing `identity:create_grant(project_id=849e4b4603a042a0886b5e4fe2d8cc38, user_id=95516d15534c4fc3aa6402a4ead06966, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:13.929707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:13.944525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1201/2420] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:13 2022] PUT /identity/v3/projects/849e4b4603a042a0886b5e4fe2d8cc38/users/95516d15534c4fc3aa6402a4ead06966/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:13.948467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:13.958379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee706db-dfe5-4251-9723-3077f0fa996e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:14.026024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88b161-aeae-4035-b7c0-b6845f3e2177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:14.026790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88b161-aeae-4035-b7c0-b6845f3e2177 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:14.027234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88b161-aeae-4035-b7c0-b6845f3e2177 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:14.027521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88b161-aeae-4035-b7c0-b6845f3e2177 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:14.028171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88b161-aeae-4035-b7c0-b6845f3e2177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:14.029876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88b161-aeae-4035-b7c0-b6845f3e2177 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:14.044292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1220/2421] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:14.045708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:14.057450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631c80f-4802-4cde-bff0-0589963c314a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:14.063237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:14.064652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:14.065036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:14.065470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] PATH_INFO: `/v3/projects/849e4b4603a042a0886b5e4fe2d8cc38/users/95516d15534c4fc3aa6402a4ead06966/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:14.067760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849e4b4603a042a0886b5e4fe2d8cc38, user_id=95516d15534c4fc3aa6402a4ead06966, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:14.070664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:14.077537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:14.083007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1202/2422] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:14 2022] PUT /identity/v3/projects/849e4b4603a042a0886b5e4fe2d8cc38/users/95516d15534c4fc3aa6402a4ead06966/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:14.093796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd739cd-50e2-4a1c-a19f-4aac533d70ab None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:14.094094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd739cd-50e2-4a1c-a19f-4aac533d70ab None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:14.094598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd739cd-50e2-4a1c-a19f-4aac533d70ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:14.136146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0bd739cd-50e2-4a1c-a19f-4aac533d70ab None None] MFA Rules not processed for user `95516d15534c4fc3aa6402a4ead06966`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:14.137433 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0bd739cd-50e2-4a1c-a19f-4aac533d70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:14.245979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1221/2423] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:14.259183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:14.261096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:14.336830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63189ad1-01d3-4e56-be66-bdfe51237453 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:14.343107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:14.343719 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:14.343994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:14.344268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:14.402922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:14.404307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:14.416054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1203/2424] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:16.078940 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303b63-74cf-4dd6-b319-caf12ad48b35 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.080229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303b63-74cf-4dd6-b319-caf12ad48b35 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.080446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303b63-74cf-4dd6-b319-caf12ad48b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.104221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-63303b63-74cf-4dd6-b319-caf12ad48b35 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:16.105987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-63303b63-74cf-4dd6-b319-caf12ad48b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.181344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1222/2425] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:14:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:16.276475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:16.278832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:16.370934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5b604-1357-436b-9f17-4636b6c82454 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:16.375754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5d72-2211-4a19-a7e7-15db250bb90c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:16.377251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5d72-2211-4a19-a7e7-15db250bb90c service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:16.377545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5d72-2211-4a19-a7e7-15db250bb90c service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:16.377867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5d72-2211-4a19-a7e7-15db250bb90c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:16.391672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a5d72-2211-4a19-a7e7-15db250bb90c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:16.394041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a5d72-2211-4a19-a7e7-15db250bb90c service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:16.404232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1204/2426] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:14:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:14:17.023306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63303b63-74cf-4dd6-b319-caf12ad48b35 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.038092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:17.123062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63303b63-74cf-4dd6-b319-caf12ad48b35 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.130439 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.133974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.134463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.134735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] PATH_INFO: `/v3/users/e007e9e208ab49aab973901c7398156a` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.136386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e007e9e208ab49aab973901c7398156a)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.139047 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.211266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e007e9e208ab49aab973901c7398156a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.221608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e007e9e208ab49aab973901c7398156a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.222483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e007e9e208ab49aab973901c7398156a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.236454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e007e9e208ab49aab973901c7398156a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.246918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e007e9e208ab49aab973901c7398156a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.256448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1223/2427] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:17 2022] DELETE /identity/v3/users/e007e9e208ab49aab973901c7398156a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:17.258608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564a990-356d-45d4-8b65-927463474922 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.258997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564a990-356d-45d4-8b65-927463474922 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.259572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564a990-356d-45d4-8b65-927463474922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.293396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0564a990-356d-45d4-8b65-927463474922 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:17.294696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0564a990-356d-45d4-8b65-927463474922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:17.346859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:17.369783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1205/2428] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:17.380210 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.381722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:17.453202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bb8ce-99f7-4965-8206-792300186759 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.457242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.457858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.458141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.458417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.469298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.471624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.485089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1224/2429] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:17.835559 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564a990-356d-45d4-8b65-927463474922 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.839122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0564a990-356d-45d4-8b65-927463474922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:17.856268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564a990-356d-45d4-8b65-927463474922 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:17.862286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:17.863391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:17.863703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:17.864098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] PATH_INFO: `/v3/projects/606b72ef709847d9a8f3a9a822f4eeac` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:17.865585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=606b72ef709847d9a8f3a9a822f4eeac)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:17.867896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:17.946207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606b72ef709847d9a8f3a9a822f4eeac', 'request_id': 'req-bec48388-8cf5-480f-87ed-052ff27707bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.953457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606b72ef709847d9a8f3a9a822f4eeac', 'request_id': 'req-bec48388-8cf5-480f-87ed-052ff27707bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.967603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606b72ef709847d9a8f3a9a822f4eeac', 'request_id': 'req-bec48388-8cf5-480f-87ed-052ff27707bc'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.981250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] The token cache is being invalidate because project 606b72ef709847d9a8f3a9a822f4eeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:17.981534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.982188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:17.988784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1206/2430] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:17 2022] DELETE /identity/v3/projects/606b72ef709847d9a8f3a9a822f4eeac => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:17.989638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:17.999913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797154d5-d539-4a6e-a88e-6e572fba1d7b service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:18.031468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:18.075509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:18.076471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:18.076920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:18.077193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] PATH_INFO: `/v3/users/95516d15534c4fc3aa6402a4ead06966` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:18.078391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=95516d15534c4fc3aa6402a4ead06966)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:18.080872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:18.143136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95516d15534c4fc3aa6402a4ead06966'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.155091 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95516d15534c4fc3aa6402a4ead06966'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.155996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95516d15534c4fc3aa6402a4ead06966'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.185998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95516d15534c4fc3aa6402a4ead06966'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.202647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ba313461-339d-4753-9cda-6b04b87392ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95516d15534c4fc3aa6402a4ead06966'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.227827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1225/2431] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:17 2022] DELETE /identity/v3/users/95516d15534c4fc3aa6402a4ead06966 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:18.563663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:18.565297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:18.587585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec48388-8cf5-480f-87ed-052ff27707bc admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:18.669310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:18.669310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:18.669310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:18.669782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] PATH_INFO: `/v3/projects/849e4b4603a042a0886b5e4fe2d8cc38` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:18.671317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849e4b4603a042a0886b5e4fe2d8cc38)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:18.673383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:18.728369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849e4b4603a042a0886b5e4fe2d8cc38', 'request_id': 'req-32b366c8-c7b3-4619-a2da-616dae1516b9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.735312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849e4b4603a042a0886b5e4fe2d8cc38', 'request_id': 'req-32b366c8-c7b3-4619-a2da-616dae1516b9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.746989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849e4b4603a042a0886b5e4fe2d8cc38', 'request_id': 'req-32b366c8-c7b3-4619-a2da-616dae1516b9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.755122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:18.762154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] The token cache is being invalidate because project 849e4b4603a042a0886b5e4fe2d8cc38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:18.762598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.763674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-32b366c8-c7b3-4619-a2da-616dae1516b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:18.766003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1207/2432] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:18 2022] DELETE /identity/v3/projects/849e4b4603a042a0886b5e4fe2d8cc38 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:18.771354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecfb69-214c-4803-b2e7-f5682e14c0b1 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:18.771664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecfb69-214c-4803-b2e7-f5682e14c0b1 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:18.771937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecfb69-214c-4803-b2e7-f5682e14c0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:18.796861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-89ecfb69-214c-4803-b2e7-f5682e14c0b1 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:18.798155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-89ecfb69-214c-4803-b2e7-f5682e14c0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:18.901641 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1226/2433] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:18.910829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb6ff-d7d3-405c-8bad-912d756826a1 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:18.911184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb6ff-d7d3-405c-8bad-912d756826a1 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:18.911440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb6ff-d7d3-405c-8bad-912d756826a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:18.941750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5a7fb6ff-d7d3-405c-8bad-912d756826a1 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:18.943172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5a7fb6ff-d7d3-405c-8bad-912d756826a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:18.964342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1208/2434] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:18.965974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecfb69-214c-4803-b2e7-f5682e14c0b1 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:18.982505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecfb69-214c-4803-b2e7-f5682e14c0b1 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:18.990960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:18.991652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:18.991936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:18.992215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:18.992913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:18.993970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:18.995872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.025062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1227/2435] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:19.026148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb6ff-d7d3-405c-8bad-912d756826a1 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.043351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb6ff-d7d3-405c-8bad-912d756826a1 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.051224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.051868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.052210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.052472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.053127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.055239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.094333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1209/2436] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:19 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:19.096326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.107200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2e9bf-d3c8-4eb9-b7c9-5899093a35ab admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.144376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:19.306177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.306797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.307099 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.307372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.308002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.309935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.351856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1228/2437] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:19 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:19.361617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.378764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211ea08-e86a-43b1-8d41-daac7d0c32e0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.383577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.384190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.384470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.384744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.385391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.386991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.397909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1210/2438] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:19.402141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.414700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cc203-b3df-40b6-a295-148ddfd86459 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.421336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5fbd7-1085-4043-87c2-ca010d53de75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.421953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5fbd7-1085-4043-87c2-ca010d53de75 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.422223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5fbd7-1085-4043-87c2-ca010d53de75 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.422495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5fbd7-1085-4043-87c2-ca010d53de75 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.423349 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5fbd7-1085-4043-87c2-ca010d53de75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.425344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5fbd7-1085-4043-87c2-ca010d53de75 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.440489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1229/2439] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:19.450266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.463933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b44e2-124f-4095-812b-39baf6e64e81 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.469310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.470656 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.471011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.471428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] PATH_INFO: `/v3/projects/77e41c96c32a4609807d58efc196029a/users/135a277344914a3b8ced2eccfa424063/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.488673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e41c96c32a4609807d58efc196029a, user_id=135a277344914a3b8ced2eccfa424063, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.492480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.507841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1211/2440] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:19 2022] PUT /identity/v3/projects/77e41c96c32a4609807d58efc196029a/users/135a277344914a3b8ced2eccfa424063/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:19.577207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f608ff-b9fb-4a16-ad9c-d5be21bc1fad None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.577705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f608ff-b9fb-4a16-ad9c-d5be21bc1fad None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.578040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f608ff-b9fb-4a16-ad9c-d5be21bc1fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.618890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f3f608ff-b9fb-4a16-ad9c-d5be21bc1fad None None] MFA Rules not processed for user `135a277344914a3b8ced2eccfa424063`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:19.621002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f3f608ff-b9fb-4a16-ad9c-d5be21bc1fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:19.715971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1230/2441] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:19.721201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.797287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5ea84-8799-44d7-bcd4-d566a215c0fe admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.802927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.804308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.805341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.806496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.807334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.809367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.865452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1212/2442] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:19 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:19.866545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f608ff-b9fb-4a16-ad9c-d5be21bc1fad None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.880574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f608ff-b9fb-4a16-ad9c-d5be21bc1fad None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:19.947839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:19.948959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:19.948959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:19.949206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:19.949656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:19.951648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:19.983107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1231/2443] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:19 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:19.985265 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:19.996991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c6d5c-0a0a-4d24-971f-06c57fba2fad admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.003236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.003932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.004305 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.004651 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.005376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.007281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.027428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1213/2444] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:14:20.028378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.046956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3753a-0054-4732-876f-ca476b31e658 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.052575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e94b9-d527-4937-9c67-daad23559e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.053270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e94b9-d527-4937-9c67-daad23559e5b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.053658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e94b9-d527-4937-9c67-daad23559e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.054031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e94b9-d527-4937-9c67-daad23559e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.054830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e94b9-d527-4937-9c67-daad23559e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.056714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e94b9-d527-4937-9c67-daad23559e5b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.073772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1232/2445] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:20.078278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.099292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086d201-e806-4d93-8a49-acf7ffd5534e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.105317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.106201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.106582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.107036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] PATH_INFO: `/v3/projects/07f8b5f5462c4bbdb4101a40e288f821/users/5e48292f79c2445aa989ebd0a2969b66/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.132851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f8b5f5462c4bbdb4101a40e288f821, user_id=5e48292f79c2445aa989ebd0a2969b66, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.135859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.164395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1214/2446] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:20 2022] PUT /identity/v3/projects/07f8b5f5462c4bbdb4101a40e288f821/users/5e48292f79c2445aa989ebd0a2969b66/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:20.165942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d0fb3-1db1-4b80-aa5e-5a30a5c6b9af None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.166861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d0fb3-1db1-4b80-aa5e-5a30a5c6b9af None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.167387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d0fb3-1db1-4b80-aa5e-5a30a5c6b9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.219351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c15d0fb3-1db1-4b80-aa5e-5a30a5c6b9af None None] MFA Rules not processed for user `5e48292f79c2445aa989ebd0a2969b66`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:20.221823 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c15d0fb3-1db1-4b80-aa5e-5a30a5c6b9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:20.239110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:20.314971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1233/2447] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:20.319478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.374733 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:20.396813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78a263-bf6d-4eb5-bb9a-59ba3fd5b393 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.402446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.403033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.403338 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.403608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.404255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.406168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.437921 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1215/2448] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:20 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:20.442419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d0fb3-1db1-4b80-aa5e-5a30a5c6b9af None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.458322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d0fb3-1db1-4b80-aa5e-5a30a5c6b9af None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.538476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.539169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.539459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.539739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.540406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.542605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.572978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1234/2449] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:20 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:20.576824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.588811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d774-f293-4296-aa6b-a3a4cc1d0f13 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.593852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.594467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.594745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.595867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.596515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.598109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.613863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1216/2450] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:20.615189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.625731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a0884-1f20-4679-8b86-1f366d436a53 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.630888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.631626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.631898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.632167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] PATH_INFO: `/v3/projects/d8a09998f5a84459a2be33a1b3626488/users/76ffe889329e42f5a3881b21a1364e48/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.649595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a09998f5a84459a2be33a1b3626488, user_id=76ffe889329e42f5a3881b21a1364e48, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.652329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.669621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1235/2451] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:20 2022] PUT /identity/v3/projects/d8a09998f5a84459a2be33a1b3626488/users/76ffe889329e42f5a3881b21a1364e48/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:20.674430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.686118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f2186-0d50-4870-8d16-7b59f7bb434c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.761809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f22e09-22af-4970-b92d-97b657b85b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.762512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f22e09-22af-4970-b92d-97b657b85b7c admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.762797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f22e09-22af-4970-b92d-97b657b85b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.763109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f22e09-22af-4970-b92d-97b657b85b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.763766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f22e09-22af-4970-b92d-97b657b85b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.768776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f22e09-22af-4970-b92d-97b657b85b7c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.782628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1217/2452] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:20.784771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.796419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38a2cd-ab8d-4190-9c5c-14d708b8f512 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:20.802025 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:20.802799 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.803170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.803514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] PATH_INFO: `/v3/projects/d8a09998f5a84459a2be33a1b3626488/users/76ffe889329e42f5a3881b21a1364e48/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.807497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a09998f5a84459a2be33a1b3626488, user_id=76ffe889329e42f5a3881b21a1364e48, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:20.808618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:20.820380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1236/2453] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:20 2022] PUT /identity/v3/projects/d8a09998f5a84459a2be33a1b3626488/users/76ffe889329e42f5a3881b21a1364e48/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:20.832836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f33d5-50d9-4558-812a-aa909cd93141 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:20.833121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f33d5-50d9-4558-812a-aa909cd93141 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:20.833399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f33d5-50d9-4558-812a-aa909cd93141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:20.870477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-ff4f33d5-50d9-4558-812a-aa909cd93141 None None] MFA Rules not processed for user `76ffe889329e42f5a3881b21a1364e48`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:20.871895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ff4f33d5-50d9-4558-812a-aa909cd93141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:20.960442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1218/2454] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:20.972501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:20.974271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:21.066550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb7e8b-d7d2-4c7c-8e8e-706c5a3e077f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:21.070340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:21.070947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:21.071254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:21.071525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:21.092127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:21.155636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:21.157531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:21.172237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1237/2455] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:22.864910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f33d5-50d9-4558-812a-aa909cd93141 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:22.879838 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f33d5-50d9-4558-812a-aa909cd93141 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:22.882948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:22.883615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:22.883898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:22.884173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:22.942186 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:22.968895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:22.971402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:22.983859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1219/2456] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:23.780574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:23.782033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:23.794348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2022df-d54b-41de-9e07-0b28d25fa409 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:23.806870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d99603-f780-444e-8966-e7246c40b1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:23.807565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d99603-f780-444e-8966-e7246c40b1ff service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:23.807919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d99603-f780-444e-8966-e7246c40b1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:23.808202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d99603-f780-444e-8966-e7246c40b1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:23.819246 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d99603-f780-444e-8966-e7246c40b1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:23.820686 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d99603-f780-444e-8966-e7246c40b1ff service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:23.837835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1238/2457] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:27.990978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:28.054578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70b4b-bb3d-4c5b-8fe5-75b79e850062 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:28.059207 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:28.059837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:28.060120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:28.060394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] PATH_INFO: `/v3/users/135a277344914a3b8ced2eccfa424063` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:28.061520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135a277344914a3b8ced2eccfa424063)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:28.063473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:28.110226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135a277344914a3b8ced2eccfa424063'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.122958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135a277344914a3b8ced2eccfa424063'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.123753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135a277344914a3b8ced2eccfa424063'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.124359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135a277344914a3b8ced2eccfa424063'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.131747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135a277344914a3b8ced2eccfa424063'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.145072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1220/2458] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:27 2022] DELETE /identity/v3/users/135a277344914a3b8ced2eccfa424063 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:28.149522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:28.150008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:28.150111 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:28.168624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:28.169803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:28.230532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1239/2459] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:28.240775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:28.242289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:28.303498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6994-6191-43ee-99af-28477d2d8c84 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:28.307300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:28.308096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:28.308217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:28.308504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:28.318595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:28.320058 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:28.325337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:28.330769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1221/2460] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:28.600329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:28.601938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:28.615056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441361d-0b5f-4e10-aa82-f9ded469880f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:28.620105 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:28.620824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:28.621107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:28.621380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] PATH_INFO: `/v3/projects/77e41c96c32a4609807d58efc196029a` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:28.622868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e41c96c32a4609807d58efc196029a)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:28.624429 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:28.676112 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e41c96c32a4609807d58efc196029a', 'request_id': 'req-98e62344-adf7-43d1-b7d2-7bedd75daea2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.682322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e41c96c32a4609807d58efc196029a', 'request_id': 'req-98e62344-adf7-43d1-b7d2-7bedd75daea2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.688318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e41c96c32a4609807d58efc196029a', 'request_id': 'req-98e62344-adf7-43d1-b7d2-7bedd75daea2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.703156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] The token cache is being invalidate because project 77e41c96c32a4609807d58efc196029a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:28.703559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.704345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.706517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1240/2461] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:28 2022] DELETE /identity/v3/projects/77e41c96c32a4609807d58efc196029a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:28.710658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:28.721160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a7f4-5b6d-4e2b-bc1b-31059ed53a95 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:28.784353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:28.785048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:28.785347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:28.785637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] PATH_INFO: `/v3/users/5e48292f79c2445aa989ebd0a2969b66` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:28.786827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e48292f79c2445aa989ebd0a2969b66)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:28.788917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:28.881977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e48292f79c2445aa989ebd0a2969b66'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.901448 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e48292f79c2445aa989ebd0a2969b66'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.902729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e48292f79c2445aa989ebd0a2969b66'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.906682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e48292f79c2445aa989ebd0a2969b66'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.917261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e48292f79c2445aa989ebd0a2969b66'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:28.934230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1222/2462] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:28 2022] DELETE /identity/v3/users/5e48292f79c2445aa989ebd0a2969b66 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:29.312203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:29.313674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:29.329014 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62344-adf7-43d1-b7d2-7bedd75daea2 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:29.367950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:29.411983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:29.412919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:29.413339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:29.413773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] PATH_INFO: `/v3/projects/07f8b5f5462c4bbdb4101a40e288f821` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:29.415927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f8b5f5462c4bbdb4101a40e288f821)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:29.418266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:29.479318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f8b5f5462c4bbdb4101a40e288f821', 'request_id': 'req-649cc988-ee50-4e70-97d4-00dccf0ce172'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.486787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f8b5f5462c4bbdb4101a40e288f821', 'request_id': 'req-649cc988-ee50-4e70-97d4-00dccf0ce172'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.493521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f8b5f5462c4bbdb4101a40e288f821', 'request_id': 'req-649cc988-ee50-4e70-97d4-00dccf0ce172'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.509457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] The token cache is being invalidate because project 07f8b5f5462c4bbdb4101a40e288f821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:29.509899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.510552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.512100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1241/2463] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:29 2022] DELETE /identity/v3/projects/07f8b5f5462c4bbdb4101a40e288f821 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:29.516232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:29.518455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:29.532264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5484c5-212d-42f4-88bd-545564d5e6fc admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:29.591480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:29.592120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:29.592404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:29.592684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] PATH_INFO: `/v3/users/76ffe889329e42f5a3881b21a1364e48` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:29.593847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ffe889329e42f5a3881b21a1364e48)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:29.596219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:29.619019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:29.646923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ffe889329e42f5a3881b21a1364e48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.658698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ffe889329e42f5a3881b21a1364e48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.659389 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ffe889329e42f5a3881b21a1364e48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.659987 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ffe889329e42f5a3881b21a1364e48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.666691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7228b617-0482-4213-a699-20d3b76bbea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ffe889329e42f5a3881b21a1364e48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:29.679941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1223/2464] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:29 2022] DELETE /identity/v3/users/76ffe889329e42f5a3881b21a1364e48 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:29.958821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:29.960310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:29.973362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc988-ee50-4e70-97d4-00dccf0ce172 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.033895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.034547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.034822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.035151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] PATH_INFO: `/v3/projects/d8a09998f5a84459a2be33a1b3626488` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.036263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a09998f5a84459a2be33a1b3626488)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.038168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.089529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a09998f5a84459a2be33a1b3626488', 'request_id': 'req-a5878189-887f-475a-bac0-cdc7fb6ed32b'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.096519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a09998f5a84459a2be33a1b3626488', 'request_id': 'req-a5878189-887f-475a-bac0-cdc7fb6ed32b'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.100957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:30.102278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a09998f5a84459a2be33a1b3626488', 'request_id': 'req-a5878189-887f-475a-bac0-cdc7fb6ed32b'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.118490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] The token cache is being invalidate because project d8a09998f5a84459a2be33a1b3626488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:30.118798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.119397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-a5878189-887f-475a-bac0-cdc7fb6ed32b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:30.120861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1242/2465] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:29 2022] DELETE /identity/v3/projects/d8a09998f5a84459a2be33a1b3626488 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:30.126246 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f1f51-d18b-403c-bf21-ca0bba931136 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.127560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f1f51-d18b-403c-bf21-ca0bba931136 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.127710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f1f51-d18b-403c-bf21-ca0bba931136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.147852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-fd2f1f51-d18b-403c-bf21-ca0bba931136 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:30.149052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fd2f1f51-d18b-403c-bf21-ca0bba931136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.216028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1224/2466] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:30.224585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ee615-f89e-4f81-9baf-21ca7ad7510b None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.224864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ee615-f89e-4f81-9baf-21ca7ad7510b None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.225136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ee615-f89e-4f81-9baf-21ca7ad7510b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.248023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9f9ee615-f89e-4f81-9baf-21ca7ad7510b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:30.249557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9f9ee615-f89e-4f81-9baf-21ca7ad7510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.258649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1243/2467] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:30.263856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f1f51-d18b-403c-bf21-ca0bba931136 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.278841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f1f51-d18b-403c-bf21-ca0bba931136 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.285061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.285872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.286156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.286432 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.287158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:30.287466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.289130 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.305948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1225/2468] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:30.307194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ee615-f89e-4f81-9baf-21ca7ad7510b None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.317918 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ee615-f89e-4f81-9baf-21ca7ad7510b None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.323377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.324022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.324305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.324576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.325242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.327207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.358847 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1244/2469] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:30 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:30.363304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.373461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8724-d6f1-4134-8faa-fd527c15a590 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.434903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.435544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.435826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.436101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.436735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.438654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.465089 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1226/2470] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:30 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:30.466605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.477286 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0610-e402-46e6-a5bf-ed6b5dae983c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.482333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.482942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.483260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.483526 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.484301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.485947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.496586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1245/2471] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:30.500823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.511740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61343be3-af45-4e01-8b8c-7b7f6fae2621 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.516438 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f2411-9e32-4c2b-bd0d-88ac203e3f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.517767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f2411-9e32-4c2b-bd0d-88ac203e3f1c admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.518049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f2411-9e32-4c2b-bd0d-88ac203e3f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.518401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f2411-9e32-4c2b-bd0d-88ac203e3f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.519373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f2411-9e32-4c2b-bd0d-88ac203e3f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.520959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f2411-9e32-4c2b-bd0d-88ac203e3f1c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.531980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1227/2472] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:30.536908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.547353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f15172-70a2-494b-9dea-8ddba5b87a9a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.552723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.553384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.553691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.553965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] PATH_INFO: `/v3/projects/abcda882db7046eab245492bc4c4d7e6/users/bccc502170e349e0848922712c574ca7/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.569994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abcda882db7046eab245492bc4c4d7e6, user_id=bccc502170e349e0848922712c574ca7, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.572730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.585605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1246/2473] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:30 2022] PUT /identity/v3/projects/abcda882db7046eab245492bc4c4d7e6/users/bccc502170e349e0848922712c574ca7/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:30.590758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d59c0-523a-4fba-94c6-883fdd422406 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.590994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d59c0-523a-4fba-94c6-883fdd422406 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.591345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d59c0-523a-4fba-94c6-883fdd422406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.599989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:30.612549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-169d59c0-523a-4fba-94c6-883fdd422406 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:30.613780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-169d59c0-523a-4fba-94c6-883fdd422406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.674255 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1228/2474] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:30.683932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:30.685617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:30.743310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f1015-d339-4958-9f6c-a400a495bac7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:30.746950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:30.747581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:30.747866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:30.748138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:30.757360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:30.758780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:30.768955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1247/2475] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:14:31.708403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4200c5-e64d-4892-8b3c-13ad8a193aa9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.709005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4200c5-e64d-4892-8b3c-13ad8a193aa9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.709005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4200c5-e64d-4892-8b3c-13ad8a193aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.745917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-1e4200c5-e64d-4892-8b3c-13ad8a193aa9 None None] MFA Rules not processed for user `bccc502170e349e0848922712c574ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:31.749209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1e4200c5-e64d-4892-8b3c-13ad8a193aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:31.855975 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1229/2476] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:31.863548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:31.865233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:31.957840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55b1c7-6506-4434-bbe5-c9f31b222b46 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:31.962648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33b0e9-33ba-44e4-a20b-6d17535939fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:31.963613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33b0e9-33ba-44e4-a20b-6d17535939fa service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:31.964256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33b0e9-33ba-44e4-a20b-6d17535939fa service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:31.964937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33b0e9-33ba-44e4-a20b-6d17535939fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:31.979026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33b0e9-33ba-44e4-a20b-6d17535939fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:31.980724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33b0e9-33ba-44e4-a20b-6d17535939fa service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:31.992554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1248/2477] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:33.410652 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:33.411584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:33.412078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:33.432340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] MFA Rules not processed for user `bccc502170e349e0848922712c574ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:33.434159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:33.446498 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1230/2478] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:36.197376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878cb99-6c0d-49e8-8da0-99d7f9ac7318 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:36.197866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878cb99-6c0d-49e8-8da0-99d7f9ac7318 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:36.197962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878cb99-6c0d-49e8-8da0-99d7f9ac7318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:36.218264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-6878cb99-6c0d-49e8-8da0-99d7f9ac7318 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:36.219403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6878cb99-6c0d-49e8-8da0-99d7f9ac7318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:36.247398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:36.279645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1249/2479] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:14:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:36.292529 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:36.293911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:36.366351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632daf5d-334a-45fd-8edc-a88b36824d5d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:36.371712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61986dbe-751a-4c12-97e1-c09cfc4bb9cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:36.372473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61986dbe-751a-4c12-97e1-c09cfc4bb9cd service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:36.372757 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61986dbe-751a-4c12-97e1-c09cfc4bb9cd service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:36.373037 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61986dbe-751a-4c12-97e1-c09cfc4bb9cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:36.382120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61986dbe-751a-4c12-97e1-c09cfc4bb9cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:36.383771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61986dbe-751a-4c12-97e1-c09cfc4bb9cd service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:36.392445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1231/2480] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:14:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:14:38.478087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878cb99-6c0d-49e8-8da0-99d7f9ac7318 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:38.488992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878cb99-6c0d-49e8-8da0-99d7f9ac7318 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:38.492659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86220d7c-c759-42ca-a22a-7d823dbe0715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:38.493271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86220d7c-c759-42ca-a22a-7d823dbe0715 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:38.493654 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86220d7c-c759-42ca-a22a-7d823dbe0715 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:38.497000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86220d7c-c759-42ca-a22a-7d823dbe0715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:38.497000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-86220d7c-c759-42ca-a22a-7d823dbe0715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:38.508525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86220d7c-c759-42ca-a22a-7d823dbe0715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:38.509961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86220d7c-c759-42ca-a22a-7d823dbe0715 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:38.520136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1250/2481] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:42.743401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa53109-3401-4a85-8859-8668f2ef2fe8 None None] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:42.744327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa53109-3401-4a85-8859-8668f2ef2fe8 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:42.744607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa53109-3401-4a85-8859-8668f2ef2fe8 None None] PATH_INFO: `/` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:42.745957 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1232/2482] 10.222.0.94 () {54 vars in 921 bytes} [Mon Aug 29 19:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 19:14:42.770102 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c966746-441c-4cc8-ac33-17b33e7b7303 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:42.770684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c966746-441c-4cc8-ac33-17b33e7b7303 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:42.770684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c966746-441c-4cc8-ac33-17b33e7b7303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:42.792848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9c966746-441c-4cc8-ac33-17b33e7b7303 None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:42.794083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9c966746-441c-4cc8-ac33-17b33e7b7303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:42.871471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa53109-3401-4a85-8859-8668f2ef2fe8 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:42.874121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4aa53109-3401-4a85-8859-8668f2ef2fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:42.911594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa53109-3401-4a85-8859-8668f2ef2fe8 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:42.922767 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1251/2483] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Aug 29 19:14:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:42.930109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:42.930754 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:42.931041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:42.931501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] PATH_INFO: `/v3/users/624a0a90866446439918bd65ce5ddc60` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:42.932728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624a0a90866446439918bd65ce5ddc60)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:42.935024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:42.938913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c043fe-140f-4820-8d05-848796afe54a None None] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:42.939431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c043fe-140f-4820-8d05-848796afe54a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:42.939905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c043fe-140f-4820-8d05-848796afe54a None None] PATH_INFO: `/` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:42.941723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1252/2484] 10.222.0.94 () {54 vars in 961 bytes} [Mon Aug 29 19:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 19:14:42.952809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfdb32-cb18-4483-a759-4858a82a0bfe None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:42.953041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:42.953158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfdb32-cb18-4483-a759-4858a82a0bfe None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:42.953402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfdb32-cb18-4483-a759-4858a82a0bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:42.992382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e7dfdb32-cb18-4483-a759-4858a82a0bfe None None] MFA Rules not processed for user `005b6400ae0c40bd8da5b9ddcc96770b`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:42.995011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e7dfdb32-cb18-4483-a759-4858a82a0bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:43.008350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1253/2485] 10.222.0.94 () {58 vars in 1058 bytes} [Mon Aug 29 19:14:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:43.021908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfdb32-cb18-4483-a759-4858a82a0bfe None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:43.034240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624a0a90866446439918bd65ce5ddc60'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.036621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfdb32-cb18-4483-a759-4858a82a0bfe None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:43.045958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:43.046855 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.047375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.047659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.050600 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624a0a90866446439918bd65ce5ddc60'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.051459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624a0a90866446439918bd65ce5ddc60'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.052158 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624a0a90866446439918bd65ce5ddc60'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.057471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:43.059487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:43.071764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-0c6d5e15-3b4c-4a8f-95a4-26b6ee9efde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624a0a90866446439918bd65ce5ddc60'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.077369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1254/2486] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:43.096228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1233/2487] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:42 2022] DELETE /identity/v3/users/624a0a90866446439918bd65ce5ddc60 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:43.313180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:43.325494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a641a7-1dfa-4c31-b9f6-d64459f75e87 service placement] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:43.404384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:43.405024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.405305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.405585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] PATH_INFO: `/v3/projects/5dda60c835244f5f9c980ed1927d9784` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.406710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dda60c835244f5f9c980ed1927d9784)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:43.409158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:43.453825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:43.462117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dda60c835244f5f9c980ed1927d9784', 'request_id': 'req-ec6cc78a-5763-47ad-ac03-b0950e3b257e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.469230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dda60c835244f5f9c980ed1927d9784', 'request_id': 'req-ec6cc78a-5763-47ad-ac03-b0950e3b257e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.476194 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dda60c835244f5f9c980ed1927d9784', 'request_id': 'req-ec6cc78a-5763-47ad-ac03-b0950e3b257e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.492276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] The token cache is being invalidate because project 5dda60c835244f5f9c980ed1927d9784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:43.492569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.493671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ec6cc78a-5763-47ad-ac03-b0950e3b257e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:43.495212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1255/2488] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:43 2022] DELETE /identity/v3/projects/5dda60c835244f5f9c980ed1927d9784 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:43.500334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27de965-2f22-482b-b8d7-eec75c70e85a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.500702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27de965-2f22-482b-b8d7-eec75c70e85a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.501049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27de965-2f22-482b-b8d7-eec75c70e85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.524339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f27de965-2f22-482b-b8d7-eec75c70e85a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:43.525578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f27de965-2f22-482b-b8d7-eec75c70e85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:43.603738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1234/2489] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:43.612735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048cea-11b2-4047-a0df-27428a940703 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.613180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048cea-11b2-4047-a0df-27428a940703 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.613525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048cea-11b2-4047-a0df-27428a940703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.639663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-fc048cea-11b2-4047-a0df-27428a940703 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:43.641003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-fc048cea-11b2-4047-a0df-27428a940703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:43.649800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1256/2490] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:43.653580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27de965-2f22-482b-b8d7-eec75c70e85a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:43.666425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27de965-2f22-482b-b8d7-eec75c70e85a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:43.673239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:43.674587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.675397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.675889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.676753 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:43.677063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:43.678682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:43.696404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1235/2491] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:43.701178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048cea-11b2-4047-a0df-27428a940703 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:43.721540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048cea-11b2-4047-a0df-27428a940703 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:43.730687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:43.731495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.731880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.732252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.733001 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:43.735272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:43.779518 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1257/2492] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:43 2022] POST /identity/v3/projects => generated 369 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:43.784568 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:43.818488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c9353-f09e-4347-bf50-7feb789bf991 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:43.905094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:43.905708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.905989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.906264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.906905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:43.909072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:43.933795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1236/2493] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:43 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:43.938000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:43.950950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a8baa-a2f6-4269-b8fb-40c3364413c7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:43.958311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:43.958943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.959260 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.959611 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.960267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:43.961878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:43.975056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1258/2494] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:43.979628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:43.991149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41916824-48ec-4fab-97fc-b91885ee152c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:43.996067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095241a-57be-4e9d-ae95-d7a064d60ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:43.996783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095241a-57be-4e9d-ae95-d7a064d60ebd admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:43.997061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095241a-57be-4e9d-ae95-d7a064d60ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:43.997332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095241a-57be-4e9d-ae95-d7a064d60ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:43.997988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095241a-57be-4e9d-ae95-d7a064d60ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:43.999711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095241a-57be-4e9d-ae95-d7a064d60ebd admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:44.001266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:44.018834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1237/2495] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:44.020387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:44.030792 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e63cd6-b5fc-4e0a-b54d-3d47b1e232f9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:44.039137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:44.039833 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:44.040141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:44.040417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] PATH_INFO: `/v3/projects/6da02c0b0e5f4892b3c15c96a3209b62/users/3dc909fb80804fc483d12d79df5a96e9/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:44.057745 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da02c0b0e5f4892b3c15c96a3209b62, user_id=3dc909fb80804fc483d12d79df5a96e9, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:44.060731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:44.076517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1259/2496] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:44 2022] PUT /identity/v3/projects/6da02c0b0e5f4892b3c15c96a3209b62/users/3dc909fb80804fc483d12d79df5a96e9/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:44.088211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b630b9-1c1c-4734-a242-e4772ae79862 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:44.088496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b630b9-1c1c-4734-a242-e4772ae79862 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:44.088770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b630b9-1c1c-4734-a242-e4772ae79862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:44.124035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c0b630b9-1c1c-4734-a242-e4772ae79862 None None] MFA Rules not processed for user `3dc909fb80804fc483d12d79df5a96e9`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:44.124035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c0b630b9-1c1c-4734-a242-e4772ae79862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:44.213822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1238/2497] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:44.226298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:44.228192 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:44.330242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749116f-2aeb-49c6-a5f8-4f2bb03ca158 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:44.340797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fd5db-14b1-4e76-b7f1-b489fe1309cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:44.341655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd5db-14b1-4e76-b7f1-b489fe1309cc service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:44.341961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd5db-14b1-4e76-b7f1-b489fe1309cc service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:44.342238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd5db-14b1-4e76-b7f1-b489fe1309cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:44.354691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fd5db-14b1-4e76-b7f1-b489fe1309cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:44.366968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fd5db-14b1-4e76-b7f1-b489fe1309cc service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:44.381928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1260/2498] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:47.224512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b630b9-1c1c-4734-a242-e4772ae79862 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:47.291491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b630b9-1c1c-4734-a242-e4772ae79862 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:47.298969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:47.299671 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:47.299950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:47.300226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] PATH_INFO: `/v3/users/3dc909fb80804fc483d12d79df5a96e9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:47.301422 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc909fb80804fc483d12d79df5a96e9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:47.303409 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:47.356565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc909fb80804fc483d12d79df5a96e9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:47.370858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc909fb80804fc483d12d79df5a96e9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:47.371649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc909fb80804fc483d12d79df5a96e9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:47.372323 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc909fb80804fc483d12d79df5a96e9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:47.379857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc909fb80804fc483d12d79df5a96e9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:47.394076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1239/2499] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:47 2022] DELETE /identity/v3/users/3dc909fb80804fc483d12d79df5a96e9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:47.398568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:47.399117 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:47.399230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:47.421960 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:47.423208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:47.471313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:47.488142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1261/2500] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:47.498148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:47.499815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:47.507712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:47.579484 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620aed-f50f-4ff5-b0bc-92b28ab0ef22 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:47.586261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc768f8-c3c6-4847-b21b-ff9064e23bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:47.587203 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc768f8-c3c6-4847-b21b-ff9064e23bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:47.587706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc768f8-c3c6-4847-b21b-ff9064e23bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:47.588268 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc768f8-c3c6-4847-b21b-ff9064e23bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:47.600563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc768f8-c3c6-4847-b21b-ff9064e23bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:47.602012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc768f8-c3c6-4847-b21b-ff9064e23bd1 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:47.621236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1240/2501] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:48.181566 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:48.183008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:48.196436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5035cd8-f320-4cde-9a97-951f2dded569 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:48.201379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:48.202036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.202310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.202605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] PATH_INFO: `/v3/projects/6da02c0b0e5f4892b3c15c96a3209b62` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.203731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6da02c0b0e5f4892b3c15c96a3209b62)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:48.206916 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:48.267992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6da02c0b0e5f4892b3c15c96a3209b62', 'request_id': 'req-331428f3-2d43-4b20-9d75-1bb6c154bde8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:48.276413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6da02c0b0e5f4892b3c15c96a3209b62', 'request_id': 'req-331428f3-2d43-4b20-9d75-1bb6c154bde8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:48.286361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6da02c0b0e5f4892b3c15c96a3209b62', 'request_id': 'req-331428f3-2d43-4b20-9d75-1bb6c154bde8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:48.302392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] The token cache is being invalidate because project 6da02c0b0e5f4892b3c15c96a3209b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:48.302830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:48.303632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-331428f3-2d43-4b20-9d75-1bb6c154bde8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:48.305205 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1262/2502] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:48 2022] DELETE /identity/v3/projects/6da02c0b0e5f4892b3c15c96a3209b62 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:48.310418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf36c6-4131-4c55-a9c5-d58ed3b69e5e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.310418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf36c6-4131-4c55-a9c5-d58ed3b69e5e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.311020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf36c6-4131-4c55-a9c5-d58ed3b69e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.334045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-cbbf36c6-4131-4c55-a9c5-d58ed3b69e5e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:48.335615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-cbbf36c6-4131-4c55-a9c5-d58ed3b69e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:48.406035 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1241/2503] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:48.413691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f427a-3fcb-4d02-9d64-15ab58cddd37 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.414189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f427a-3fcb-4d02-9d64-15ab58cddd37 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.414350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f427a-3fcb-4d02-9d64-15ab58cddd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.434261 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-810f427a-3fcb-4d02-9d64-15ab58cddd37 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:48.435528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-810f427a-3fcb-4d02-9d64-15ab58cddd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:48.444177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1263/2504] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:48.448511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf36c6-4131-4c55-a9c5-d58ed3b69e5e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:48.459290 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf36c6-4131-4c55-a9c5-d58ed3b69e5e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:48.464072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:48.464705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.464985 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.465258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.465913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:48.466218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:48.467876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:48.481442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1242/2505] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:14:48.485773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f427a-3fcb-4d02-9d64-15ab58cddd37 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:48.495409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f427a-3fcb-4d02-9d64-15ab58cddd37 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:48.500008 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:48.503465 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.503698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.503922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.504567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:48.506482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:48.538336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1264/2506] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:48 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:48.542644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:48.554552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc66f6-0e1c-4858-a5e6-34e3b09520c9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:48.626355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:48.627162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.627463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.627734 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.628387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:48.630326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:48.655816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1243/2507] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:48 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:48.656958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:48.668096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9885444-eff6-48d1-b571-c72c3a785954 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:48.673149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:48.674167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.674354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.674623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.675289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:48.676878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:48.687224 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1265/2508] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:14:48.693021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:48.703911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5cab-0a2e-4e8e-963e-3e70620c0878 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:48.709691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39970e7-7af9-4146-ae89-3f2ea4a55e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:48.710303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39970e7-7af9-4146-ae89-3f2ea4a55e95 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.710609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39970e7-7af9-4146-ae89-3f2ea4a55e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.710883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39970e7-7af9-4146-ae89-3f2ea4a55e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.711564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39970e7-7af9-4146-ae89-3f2ea4a55e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:48.713168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39970e7-7af9-4146-ae89-3f2ea4a55e95 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:48.725640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1244/2509] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:48.729795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:48.748232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c09d40-7e48-4474-8209-46c8dcb2b3b4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:48.756124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:48.757232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.757729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.758226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] PATH_INFO: `/v3/projects/77286fee195d42aba4988763f6d40a6c/users/d0d02f30c93a4ff98195c026da31e14d/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.785156 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77286fee195d42aba4988763f6d40a6c, user_id=d0d02f30c93a4ff98195c026da31e14d, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:48.787964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:48.811664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1266/2510] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:48 2022] PUT /identity/v3/projects/77286fee195d42aba4988763f6d40a6c/users/d0d02f30c93a4ff98195c026da31e14d/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:48.822464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91da1b3-c831-464e-82f7-afcb39235b7f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:48.822750 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91da1b3-c831-464e-82f7-afcb39235b7f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:48.823104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91da1b3-c831-464e-82f7-afcb39235b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:48.856699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:48.860603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f91da1b3-c831-464e-82f7-afcb39235b7f None None] MFA Rules not processed for user `d0d02f30c93a4ff98195c026da31e14d`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:48.862680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f91da1b3-c831-464e-82f7-afcb39235b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:48.956021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1245/2511] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:48.960553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.041684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51050caa-229b-4f2e-a773-d3d6cb9df760 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.041684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.041684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.041684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.042056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.042056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.042056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.059042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:49.074708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1267/2512] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:48 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:49.078429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91da1b3-c831-464e-82f7-afcb39235b7f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.089623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91da1b3-c831-464e-82f7-afcb39235b7f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.161296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.162589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.162932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.163173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.163892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.166690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.197003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.203297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1246/2513] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:49 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:49.209508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbce574-fb11-411d-a5d4-a81ed4694169 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.215596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.216694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.217067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.219415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.220302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.222923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.235904 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1268/2514] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:49.240193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.254007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291e45e-ce48-44ab-a6c2-0049a97da9e9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.262584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.263404 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.263841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.264155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] PATH_INFO: `/v3/projects/08c5cd9acba842bb807e8cd2d9b42cf3/users/b2c8e4d5a7d145aeb9e9be6f8aea5146/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.286261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c5cd9acba842bb807e8cd2d9b42cf3, user_id=b2c8e4d5a7d145aeb9e9be6f8aea5146, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.289385 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.308616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1247/2515] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:49 2022] PUT /identity/v3/projects/08c5cd9acba842bb807e8cd2d9b42cf3/users/b2c8e4d5a7d145aeb9e9be6f8aea5146/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:49.312429 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.323958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fb152-d172-49ac-a1ff-9b8e1616f999 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.398926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162259f5-c22f-4038-be0b-7afcc63f9b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.403790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162259f5-c22f-4038-be0b-7afcc63f9b59 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.404481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162259f5-c22f-4038-be0b-7afcc63f9b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.405873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162259f5-c22f-4038-be0b-7afcc63f9b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.406795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162259f5-c22f-4038-be0b-7afcc63f9b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.410221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162259f5-c22f-4038-be0b-7afcc63f9b59 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.437484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1269/2516] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:49.442119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.456964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c832a34-202b-42ca-93ac-418620e66373 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.461897 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.462573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.462853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.463170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] PATH_INFO: `/v3/projects/08c5cd9acba842bb807e8cd2d9b42cf3/users/b2c8e4d5a7d145aeb9e9be6f8aea5146/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.465187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c5cd9acba842bb807e8cd2d9b42cf3, user_id=b2c8e4d5a7d145aeb9e9be6f8aea5146, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.467863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.479809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1248/2517] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:49 2022] PUT /identity/v3/projects/08c5cd9acba842bb807e8cd2d9b42cf3/users/b2c8e4d5a7d145aeb9e9be6f8aea5146/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:49.492249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12531c7e-0ec7-4798-9bed-b60b05519ae2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.492601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12531c7e-0ec7-4798-9bed-b60b05519ae2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.492881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12531c7e-0ec7-4798-9bed-b60b05519ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.522638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-12531c7e-0ec7-4798-9bed-b60b05519ae2 None None] MFA Rules not processed for user `b2c8e4d5a7d145aeb9e9be6f8aea5146`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:49.523880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-12531c7e-0ec7-4798-9bed-b60b05519ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:49.598142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1270/2518] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:49.609844 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.612780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:49.674889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6118-cc90-4aae-9690-09a967e1958a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.678922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.679601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.679863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.680141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.690857 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.692592 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.696909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:49.705062 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1249/2519] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:49.875383 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12531c7e-0ec7-4798-9bed-b60b05519ae2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:49.886128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12531c7e-0ec7-4798-9bed-b60b05519ae2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:49.889949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adeffdf-956e-4328-a2fd-21530268f7b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:49.890991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeffdf-956e-4328-a2fd-21530268f7b1 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:49.891319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeffdf-956e-4328-a2fd-21530268f7b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:49.891619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeffdf-956e-4328-a2fd-21530268f7b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:49.950151 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adeffdf-956e-4328-a2fd-21530268f7b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:49.951521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adeffdf-956e-4328-a2fd-21530268f7b1 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:49.960658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1271/2520] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:54.799199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:54.864806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16c399-feb0-4c79-9ec4-67198a36a798 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:54.879866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:54.880541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:54.880827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:54.881106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] PATH_INFO: `/v3/users/d0d02f30c93a4ff98195c026da31e14d` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:54.882339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d02f30c93a4ff98195c026da31e14d)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:54.884316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:54.935728 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d02f30c93a4ff98195c026da31e14d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:54.951521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d02f30c93a4ff98195c026da31e14d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:54.952906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d02f30c93a4ff98195c026da31e14d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:54.953720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d02f30c93a4ff98195c026da31e14d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:54.964426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d02f30c93a4ff98195c026da31e14d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:54.978122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1250/2521] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:54 2022] DELETE /identity/v3/users/d0d02f30c93a4ff98195c026da31e14d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:54.982798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:54.983355 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:54.983501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:54.993185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:55.005086 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:55.006399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:55.069311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1272/2522] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:55.078941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:55.080666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:55.142483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e64a-0e8c-41ee-913d-257e5e54ad0d admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:55.146504 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:55.147140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:55.147421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:55.147692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:55.157124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:55.159416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:55.169355 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1251/2523] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:14:55.468141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:55.469683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:55.482925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f467e83-92c8-428a-b5cd-5cf20b5f1514 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:55.487704 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:55.488316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:55.488587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:55.488855 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] PATH_INFO: `/v3/projects/77286fee195d42aba4988763f6d40a6c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:55.489971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77286fee195d42aba4988763f6d40a6c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:55.491915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:55.549377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77286fee195d42aba4988763f6d40a6c', 'request_id': 'req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.556150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77286fee195d42aba4988763f6d40a6c', 'request_id': 'req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.561990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77286fee195d42aba4988763f6d40a6c', 'request_id': 'req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.574885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] The token cache is being invalidate because project 77286fee195d42aba4988763f6d40a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:55.575207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.575825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad78b1d2-d2ec-4181-af35-dcebfec5afe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.577299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1273/2524] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:55 2022] DELETE /identity/v3/projects/77286fee195d42aba4988763f6d40a6c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:55.582607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:55.594571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017ddc-63f5-47c5-9fd5-d00e5fdb2c07 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:55.612094 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:55.655506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:55.656153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:55.656421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:55.656695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] PATH_INFO: `/v3/users/b2c8e4d5a7d145aeb9e9be6f8aea5146` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:55.657861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c8e4d5a7d145aeb9e9be6f8aea5146)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:55.659852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:55.706395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c8e4d5a7d145aeb9e9be6f8aea5146'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.718514 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c8e4d5a7d145aeb9e9be6f8aea5146'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.719222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c8e4d5a7d145aeb9e9be6f8aea5146'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.719913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c8e4d5a7d145aeb9e9be6f8aea5146'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.727587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c8e4d5a7d145aeb9e9be6f8aea5146'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:55.743980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1252/2525] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:55 2022] DELETE /identity/v3/users/b2c8e4d5a7d145aeb9e9be6f8aea5146 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:56.131586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.132694 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.133191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.157443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:56.159368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:56.192351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:56.195322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:56.233989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1274/2526] 10.222.0.94 () {58 vars in 967 bytes} [Mon Aug 29 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:56.248643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:56.250135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:56.281954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc2238-b352-4e38-b102-527c3ff2f805 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:56.292266 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:56.293059 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.293349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.293616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] PATH_INFO: `/v3/projects/08c5cd9acba842bb807e8cd2d9b42cf3` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.294118 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:56.294890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c5cd9acba842bb807e8cd2d9b42cf3)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:56.297939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:56.341976 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44778f-ae80-4280-a49a-b0f9f70fbd7c None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:56.348577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32d864-6aae-41d9-9ec1-9b31ad741b84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:56.349291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d864-6aae-41d9-9ec1-9b31ad741b84 service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.349529 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d864-6aae-41d9-9ec1-9b31ad741b84 service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.349759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d864-6aae-41d9-9ec1-9b31ad741b84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.354601 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:56.362173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32d864-6aae-41d9-9ec1-9b31ad741b84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:56.365211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32d864-6aae-41d9-9ec1-9b31ad741b84 service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:56.368349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c5cd9acba842bb807e8cd2d9b42cf3', 'request_id': 'req-b9fcd304-1f55-49af-bc64-b01f200f799b'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:56.381040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c5cd9acba842bb807e8cd2d9b42cf3', 'request_id': 'req-b9fcd304-1f55-49af-bc64-b01f200f799b'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:56.386953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1275/2527] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:14:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 19:14:56.397033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c5cd9acba842bb807e8cd2d9b42cf3', 'request_id': 'req-b9fcd304-1f55-49af-bc64-b01f200f799b'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:56.421869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] The token cache is being invalidate because project 08c5cd9acba842bb807e8cd2d9b42cf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:56.422420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:56.423116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b9fcd304-1f55-49af-bc64-b01f200f799b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:56.425423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1253/2528] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:56 2022] DELETE /identity/v3/projects/08c5cd9acba842bb807e8cd2d9b42cf3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:56.431134 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53769360-7da5-45cd-b33f-4b7127b00a7d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.431406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53769360-7da5-45cd-b33f-4b7127b00a7d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.431674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53769360-7da5-45cd-b33f-4b7127b00a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.468276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-53769360-7da5-45cd-b33f-4b7127b00a7d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:56.469504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-53769360-7da5-45cd-b33f-4b7127b00a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:56.563271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1276/2529] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:56.573028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741307-68a7-4594-a094-ee0c656cdfc6 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.573028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741307-68a7-4594-a094-ee0c656cdfc6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.573235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741307-68a7-4594-a094-ee0c656cdfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.608359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-96741307-68a7-4594-a094-ee0c656cdfc6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:56.609710 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-96741307-68a7-4594-a094-ee0c656cdfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:56.619779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1254/2530] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:56.623850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53769360-7da5-45cd-b33f-4b7127b00a7d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:56.640133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53769360-7da5-45cd-b33f-4b7127b00a7d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:56.656200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:56.656927 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.657318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.657665 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.658712 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:14:56.659106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:56.660928 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:56.685430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1277/2531] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:14:56.690804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96741307-68a7-4594-a094-ee0c656cdfc6 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:56.700616 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96741307-68a7-4594-a094-ee0c656cdfc6 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:56.722852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:56.723518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.723802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.724080 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.738067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:56.740590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:56.788216 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1255/2532] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:56 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:56.788687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:56.804003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc90bc-2874-4b61-b2e2-1a036514aed3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:56.888583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:56.889206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.889489 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.889761 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.890574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:56.893753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:56.929798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1278/2533] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:56 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:14:56.934525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:56.946849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cbf82-f92b-414e-ba1a-bce4a9b3ab65 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:56.951765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:56.952374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.952650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.952920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.953565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:56.955418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:56.969114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1256/2534] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:56.970327 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:56.982309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66cc1a-ed40-4e7d-bbea-eef0da386f6b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:56.988141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb8817-38d6-4223-bcdb-dddeb7dcc0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:56.988758 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb8817-38d6-4223-bcdb-dddeb7dcc0ff admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:56.989043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb8817-38d6-4223-bcdb-dddeb7dcc0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:56.989395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb8817-38d6-4223-bcdb-dddeb7dcc0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:56.989952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb8817-38d6-4223-bcdb-dddeb7dcc0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:56.991588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb8817-38d6-4223-bcdb-dddeb7dcc0ff admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.003608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1279/2535] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:57.007836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.018173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51aad-6475-47ca-89a9-74d10c2143ba admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:57.022861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:57.023563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.023851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.024133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] PATH_INFO: `/v3/projects/4864edb081354b9ebf6b151a8ab1693c/users/e34093b96b924fff8cb0fc78baed335e/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.059462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4864edb081354b9ebf6b151a8ab1693c, user_id=e34093b96b924fff8cb0fc78baed335e, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:57.062672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.100821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1257/2536] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:57 2022] PUT /identity/v3/projects/4864edb081354b9ebf6b151a8ab1693c/users/e34093b96b924fff8cb0fc78baed335e/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:57.112150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc09b99-e621-4952-85b3-321c0239509d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.112431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc09b99-e621-4952-85b3-321c0239509d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.112856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc09b99-e621-4952-85b3-321c0239509d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.153058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-abc09b99-e621-4952-85b3-321c0239509d None None] MFA Rules not processed for user `e34093b96b924fff8cb0fc78baed335e`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:57.153986 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-abc09b99-e621-4952-85b3-321c0239509d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:57.174479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:57.240606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1280/2537] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:57.245180 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.332236 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a7d2f-0ddb-4d47-af51-bfe38631b4ac admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:57.337443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:57.338078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.338457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.338764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.339472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:57.341402 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.375364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1258/2538] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:14:57 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:57.380066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc09b99-e621-4952-85b3-321c0239509d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.391766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc09b99-e621-4952-85b3-321c0239509d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:57.502756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:57.503724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.504015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.504294 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.504948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:57.506963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.536335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1281/2539] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:14:57 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:14:57.540183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.562827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae075d-e507-41a2-a61b-e630731a0aa0 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:57.567832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:57.568497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.568782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.569079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.569760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:57.571476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.584642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1259/2540] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:57.588905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.599861 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84407012-cdbf-42bf-a8b9-a2b33e7d25cf admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:57.604796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:57.605468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.605752 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.606030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] PATH_INFO: `/v3/projects/55208d86ddc14042ae6c316ec5f7627d/users/ef53913927eb4b16a5a1ba6217a03438/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.624632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=55208d86ddc14042ae6c316ec5f7627d, user_id=ef53913927eb4b16a5a1ba6217a03438, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:57.627626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.643184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1282/2541] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:57 2022] PUT /identity/v3/projects/55208d86ddc14042ae6c316ec5f7627d/users/ef53913927eb4b16a5a1ba6217a03438/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:14:57.647415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.657702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf009-9012-4394-a9c8-7d00d0f53fa1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:57.701645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:57.726209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927564f4-fe9f-47db-844e-000282007a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:57.726831 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927564f4-fe9f-47db-844e-000282007a32 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.727464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927564f4-fe9f-47db-844e-000282007a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.727464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927564f4-fe9f-47db-844e-000282007a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.728052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927564f4-fe9f-47db-844e-000282007a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:57.729645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927564f4-fe9f-47db-844e-000282007a32 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.743437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1260/2542] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:14:57.748185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.762416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867add4-7083-44e8-b316-455307e4e8ba admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:57.769880 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:57.770549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.770828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.771119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] PATH_INFO: `/v3/projects/55208d86ddc14042ae6c316ec5f7627d/users/ef53913927eb4b16a5a1ba6217a03438/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.773191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55208d86ddc14042ae6c316ec5f7627d, user_id=ef53913927eb4b16a5a1ba6217a03438, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:57.777242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:57.793270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1283/2543] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:14:57 2022] PUT /identity/v3/projects/55208d86ddc14042ae6c316ec5f7627d/users/ef53913927eb4b16a5a1ba6217a03438/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:57.806765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ed526-f3d3-4d95-9bae-ed9e38d1f02a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:57.807087 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ed526-f3d3-4d95-9bae-ed9e38d1f02a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:57.807394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ed526-f3d3-4d95-9bae-ed9e38d1f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:57.850742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-239ed526-f3d3-4d95-9bae-ed9e38d1f02a None None] MFA Rules not processed for user `ef53913927eb4b16a5a1ba6217a03438`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:57.852048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-239ed526-f3d3-4d95-9bae-ed9e38d1f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:57.926692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1261/2544] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:14:57.938050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:57.940374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:58.002684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6fe7b-5418-46f5-ac24-28c22587b240 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:58.006392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120f02f-29ef-45b5-b24c-fc7bef3d68f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:58.007004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120f02f-29ef-45b5-b24c-fc7bef3d68f6 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:58.007310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120f02f-29ef-45b5-b24c-fc7bef3d68f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:58.007590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120f02f-29ef-45b5-b24c-fc7bef3d68f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:58.016905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120f02f-29ef-45b5-b24c-fc7bef3d68f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:58.019063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120f02f-29ef-45b5-b24c-fc7bef3d68f6 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:58.030399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1284/2545] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:58.451886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ed526-f3d3-4d95-9bae-ed9e38d1f02a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:58.541315 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ed526-f3d3-4d95-9bae-ed9e38d1f02a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:58.547344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:58.548152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:58.548560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:58.549589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] PATH_INFO: `/v3/users/e34093b96b924fff8cb0fc78baed335e` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:58.550867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34093b96b924fff8cb0fc78baed335e)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:58.553099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:58.619256 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34093b96b924fff8cb0fc78baed335e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:58.632230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:58.634654 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34093b96b924fff8cb0fc78baed335e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:58.635414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34093b96b924fff8cb0fc78baed335e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:58.636595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34093b96b924fff8cb0fc78baed335e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:58.646195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34093b96b924fff8cb0fc78baed335e'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:58.669237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1262/2546] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:58 2022] DELETE /identity/v3/users/e34093b96b924fff8cb0fc78baed335e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:58.671187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:58.671672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:58.672791 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:58.701958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:14:58.703200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:58.712795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:58.774270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1285/2547] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:14:58.784557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:58.786015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:58.849319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300edab7-7507-4c3b-ad51-c21b48d0c323 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:58.854122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:58.854743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:58.855027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:58.855396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:58.867230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:58.868527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:58.887855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1263/2548] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:14:59.237281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:59.239041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:59.270753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7191d1-5cc6-4386-a131-449d8015e5df None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:59.277926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:59.278659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:59.279039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:59.279301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] PATH_INFO: `/v3/projects/4864edb081354b9ebf6b151a8ab1693c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:59.280390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4864edb081354b9ebf6b151a8ab1693c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:59.284164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:59.351491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4864edb081354b9ebf6b151a8ab1693c', 'request_id': 'req-ad09e54d-70e7-41f8-af0e-df185785eee8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.359499 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4864edb081354b9ebf6b151a8ab1693c', 'request_id': 'req-ad09e54d-70e7-41f8-af0e-df185785eee8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.368885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4864edb081354b9ebf6b151a8ab1693c', 'request_id': 'req-ad09e54d-70e7-41f8-af0e-df185785eee8'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.382237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] The token cache is being invalidate because project 4864edb081354b9ebf6b151a8ab1693c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:59.382860 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.383895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.385428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1286/2549] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:59 2022] DELETE /identity/v3/projects/4864edb081354b9ebf6b151a8ab1693c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:59.389446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:59.401194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e52d2-4454-4f65-8c19-4f64591a9efa service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:59.465597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:59.466332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:59.466681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:59.467175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] PATH_INFO: `/v3/users/ef53913927eb4b16a5a1ba6217a03438` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:59.468314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef53913927eb4b16a5a1ba6217a03438)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:59.470247 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:59.523025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef53913927eb4b16a5a1ba6217a03438'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.536435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef53913927eb4b16a5a1ba6217a03438'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.537173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef53913927eb4b16a5a1ba6217a03438'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.537775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef53913927eb4b16a5a1ba6217a03438'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.545557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6d1132f2-3a03-42d7-a473-3dcfc4ecc28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef53913927eb4b16a5a1ba6217a03438'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.560673 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1264/2550] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:14:59 2022] DELETE /identity/v3/users/ef53913927eb4b16a5a1ba6217a03438 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:14:59.842426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:14:59.843991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:14:59.858536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e54d-70e7-41f8-af0e-df185785eee8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:14:59.916683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:14:59.917228 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:14:59.917513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:14:59.917784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] PATH_INFO: `/v3/projects/55208d86ddc14042ae6c316ec5f7627d` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:14:59.918893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55208d86ddc14042ae6c316ec5f7627d)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:14:59.921354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:14:59.962094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:14:59.970919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55208d86ddc14042ae6c316ec5f7627d', 'request_id': 'req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.977243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55208d86ddc14042ae6c316ec5f7627d', 'request_id': 'req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.982862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55208d86ddc14042ae6c316ec5f7627d', 'request_id': 'req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.996101 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] The token cache is being invalidate because project 55208d86ddc14042ae6c316ec5f7627d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:14:59.996385 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.997009 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d4486c4a-9df0-4e74-b9c4-f8a90182fc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:14:59.998490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1287/2551] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:14:59 2022] DELETE /identity/v3/projects/55208d86ddc14042ae6c316ec5f7627d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:00.005006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4215f2-44c9-4265-ae4e-2db922c12d8d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.005299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4215f2-44c9-4265-ae4e-2db922c12d8d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.005578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4215f2-44c9-4265-ae4e-2db922c12d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.014190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:00.030351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5e4215f2-44c9-4265-ae4e-2db922c12d8d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:00.031615 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5e4215f2-44c9-4265-ae4e-2db922c12d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:00.102564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1265/2552] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:00.110520 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c4a46-874c-4be3-b26a-93977e800463 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.110806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c4a46-874c-4be3-b26a-93977e800463 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.111090 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c4a46-874c-4be3-b26a-93977e800463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.132660 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-0c1c4a46-874c-4be3-b26a-93977e800463 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:00.133912 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-0c1c4a46-874c-4be3-b26a-93977e800463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:00.141875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1288/2553] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:15:00.146223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4215f2-44c9-4265-ae4e-2db922c12d8d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.156512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4215f2-44c9-4265-ae4e-2db922c12d8d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.161386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.162063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.162348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.162663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.163359 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:00.163663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.165332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.179749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1266/2554] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:00.184067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c4a46-874c-4be3-b26a-93977e800463 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.194225 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c4a46-874c-4be3-b26a-93977e800463 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.198957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.199595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.199875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.200144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.200804 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.202729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.232854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1289/2555] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:00 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:00.243715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.256500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb3d1f-2be6-4ff0-878d-9b626f1594f9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.320340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.320955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.321232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.321502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.322136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.324298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.353597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1267/2556] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:00 2022] POST /identity/v3/users => generated 280 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:00.354863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.365656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb69600-60e4-4274-b9ab-8ee4475e62b4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.370387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.370988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.371292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.371577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.372181 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.373844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.385083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1290/2557] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:00.389096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.399289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0805735-9971-4f6f-b25a-23b405d98404 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.403944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab25fd-f061-419c-a63c-713c09d1cbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.404547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab25fd-f061-419c-a63c-713c09d1cbea admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.404827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab25fd-f061-419c-a63c-713c09d1cbea admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.405098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab25fd-f061-419c-a63c-713c09d1cbea admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.405745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab25fd-f061-419c-a63c-713c09d1cbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.407358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab25fd-f061-419c-a63c-713c09d1cbea admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.418772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1268/2558] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:15:00.423281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.434103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d117c0-9fed-402f-b7fc-59136996603c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.438569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.439243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.439737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.440059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] PATH_INFO: `/v3/projects/c463e13e5a8648629fcd878137990dcc/users/4aca44d38201431e8bde95a2098f2467/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.454963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c463e13e5a8648629fcd878137990dcc, user_id=4aca44d38201431e8bde95a2098f2467, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.459643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.475438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1291/2559] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:00 2022] PUT /identity/v3/projects/c463e13e5a8648629fcd878137990dcc/users/4aca44d38201431e8bde95a2098f2467/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:00.487476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7399bc0-d7a9-4c33-8e5a-ec9d91d3db0b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.488032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7399bc0-d7a9-4c33-8e5a-ec9d91d3db0b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.488133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7399bc0-d7a9-4c33-8e5a-ec9d91d3db0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.517456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f7399bc0-d7a9-4c33-8e5a-ec9d91d3db0b None None] MFA Rules not processed for user `4aca44d38201431e8bde95a2098f2467`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:00.518950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f7399bc0-d7a9-4c33-8e5a-ec9d91d3db0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:00.591360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:00.600431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1269/2560] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:00.638524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.640477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:00.710437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e95144-6be3-45dc-96ad-4327c51f065e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.714004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68132c3e-a3b8-4c28-84a0-aedf61e18a6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.714610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68132c3e-a3b8-4c28-84a0-aedf61e18a6d service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.714887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68132c3e-a3b8-4c28-84a0-aedf61e18a6d service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.715195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68132c3e-a3b8-4c28-84a0-aedf61e18a6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.724664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68132c3e-a3b8-4c28-84a0-aedf61e18a6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.726146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68132c3e-a3b8-4c28-84a0-aedf61e18a6d service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.735402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1292/2561] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:00.750898 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7399bc0-d7a9-4c33-8e5a-ec9d91d3db0b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:00.834685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7399bc0-d7a9-4c33-8e5a-ec9d91d3db0b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:00.841084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:00.841826 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.842201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.842545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] PATH_INFO: `/v3/users/4aca44d38201431e8bde95a2098f2467` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.843791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aca44d38201431e8bde95a2098f2467)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:00.846425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:00.900170 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aca44d38201431e8bde95a2098f2467'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:00.913370 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aca44d38201431e8bde95a2098f2467'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:00.914140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aca44d38201431e8bde95a2098f2467'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:00.914765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aca44d38201431e8bde95a2098f2467'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:00.921752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aca44d38201431e8bde95a2098f2467'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:00.936496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1270/2562] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:00 2022] DELETE /identity/v3/users/4aca44d38201431e8bde95a2098f2467 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:00.941763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:00.942226 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:00.942324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:00.962436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:00.963609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:01.027955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1293/2563] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:01.037686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:01.039159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:01.098345 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ac7ee-4d44-4d86-99e8-ff3b1ee85133 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:01.102053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ef50e-3af2-415c-9351-ddba5f25cfff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:01.102669 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ef50e-3af2-415c-9351-ddba5f25cfff service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:01.102953 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ef50e-3af2-415c-9351-ddba5f25cfff service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:01.103262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ef50e-3af2-415c-9351-ddba5f25cfff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:01.115198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ef50e-3af2-415c-9351-ddba5f25cfff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:01.118377 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ef50e-3af2-415c-9351-ddba5f25cfff service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:01.131429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1271/2564] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:15:01.564120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:01.565957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:01.579451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c32f9f-5ae4-4c00-b66e-cd5da6d90984 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:01.584984 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:01.585630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:01.585909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:01.586177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] PATH_INFO: `/v3/projects/c463e13e5a8648629fcd878137990dcc` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:01.587404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c463e13e5a8648629fcd878137990dcc)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:01.589321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:01.591266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:01.645632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c463e13e5a8648629fcd878137990dcc', 'request_id': 'req-1c857750-f156-4acb-a9d6-58289255181d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:01.652172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c463e13e5a8648629fcd878137990dcc', 'request_id': 'req-1c857750-f156-4acb-a9d6-58289255181d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:01.658290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c463e13e5a8648629fcd878137990dcc', 'request_id': 'req-1c857750-f156-4acb-a9d6-58289255181d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:01.675548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] The token cache is being invalidate because project c463e13e5a8648629fcd878137990dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:01.675852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:01.676581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1c857750-f156-4acb-a9d6-58289255181d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:01.678297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1294/2565] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:01 2022] DELETE /identity/v3/projects/c463e13e5a8648629fcd878137990dcc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:01.683118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a626cf-bc94-4be5-8823-545d52b5fabf None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:01.683612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a626cf-bc94-4be5-8823-545d52b5fabf None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:01.683720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a626cf-bc94-4be5-8823-545d52b5fabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:01.707692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-91a626cf-bc94-4be5-8823-545d52b5fabf None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:01.709150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-91a626cf-bc94-4be5-8823-545d52b5fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:01.724658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:01.780964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1272/2566] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:01.789425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072999d-66d7-413d-99b0-6704e7cf6360 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:01.789730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072999d-66d7-413d-99b0-6704e7cf6360 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:01.790128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072999d-66d7-413d-99b0-6704e7cf6360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:01.814857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d072999d-66d7-413d-99b0-6704e7cf6360 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:01.816104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d072999d-66d7-413d-99b0-6704e7cf6360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:01.823971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1295/2567] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:01.828470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a626cf-bc94-4be5-8823-545d52b5fabf None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:01.841358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a626cf-bc94-4be5-8823-545d52b5fabf None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:01.847386 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:01.848011 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:01.848289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:01.848561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:01.849537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:01.849537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:01.851156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:01.872322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1273/2568] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:01.878110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072999d-66d7-413d-99b0-6704e7cf6360 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:01.888812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072999d-66d7-413d-99b0-6704e7cf6360 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:01.894444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:01.895322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:01.895655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:01.896255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:01.897072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:01.898890 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:01.932567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1296/2569] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:01 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:01.938643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:01.953106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8807338-27ac-43c4-a053-35c08fd7c4d1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:02.025778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:02.026408 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.026929 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.027293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.028520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:02.030041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:02.066871 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1274/2570] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:01 2022] POST /identity/v3/users => generated 285 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:02.068109 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:02.081276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6bc9a-7bcc-48a4-b990-3699cac2eaad admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:02.087287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:02.087905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.088186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.088460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.089110 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:02.090856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:02.103467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1297/2571] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:15:02.107503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:02.117955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158227d-bcb8-49f3-a842-fb96ca09a448 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:02.123104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb3cb3-79dd-410f-a734-1e19206d3350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:02.123960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb3cb3-79dd-410f-a734-1e19206d3350 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.124044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb3cb3-79dd-410f-a734-1e19206d3350 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.124317 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb3cb3-79dd-410f-a734-1e19206d3350 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.124969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb3cb3-79dd-410f-a734-1e19206d3350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:02.126544 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb3cb3-79dd-410f-a734-1e19206d3350 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:02.140052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1275/2572] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:15:02.144885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:02.155398 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea70cc-8a16-429b-b251-c268c852f862 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:02.161141 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:02.161802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.162182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.162437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] PATH_INFO: `/v3/projects/0c47352146d540369a0b4079a5ba9e8f/users/30cd65c8b64d4a6e955d678096e4e927/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.178325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c47352146d540369a0b4079a5ba9e8f, user_id=30cd65c8b64d4a6e955d678096e4e927, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:02.181150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:02.195004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1298/2573] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:02 2022] PUT /identity/v3/projects/0c47352146d540369a0b4079a5ba9e8f/users/30cd65c8b64d4a6e955d678096e4e927/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:02.206367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ab242-bd19-4e5a-9e14-062cdb4abc1a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.206675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ab242-bd19-4e5a-9e14-062cdb4abc1a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.206977 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ab242-bd19-4e5a-9e14-062cdb4abc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.236870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-5d7ab242-bd19-4e5a-9e14-062cdb4abc1a None None] MFA Rules not processed for user `30cd65c8b64d4a6e955d678096e4e927`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:02.238099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5d7ab242-bd19-4e5a-9e14-062cdb4abc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:02.307611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1276/2574] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:02.319252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:02.320948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:02.381280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04526b16-eb0d-48d3-8fdf-42797c26175b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:02.385606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:02.386256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:02.386562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:02.386885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:02.396540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:02.398179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:02.409018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1299/2575] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:03.019055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:03.019564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:03.019707 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:03.042049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] MFA Rules not processed for user `bccc502170e349e0848922712c574ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:03.043462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:03.055107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:03.104961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1277/2576] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:04.567597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:04.569076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:04.602961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:04.631249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858d884-fa00-42c2-90fe-33a54e011fb0 service glance] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:04.634898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:04.635504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:04.635784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:04.636059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:04.637797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:04.702413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:04.703891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:04.714264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1300/2577] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:09.244233 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:09.247530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:09.269495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293b501-51af-483a-a3d7-5795a6e62c2c None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:09.274193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:09.275021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:09.275329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:09.275785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] PATH_INFO: `/v3/users/ac0bc9e010d044a5a2147010cbdb4d85` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:09.276960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0bc9e010d044a5a2147010cbdb4d85)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:09.279109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:09.341447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0bc9e010d044a5a2147010cbdb4d85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.357509 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0bc9e010d044a5a2147010cbdb4d85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.358971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0bc9e010d044a5a2147010cbdb4d85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.358971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0bc9e010d044a5a2147010cbdb4d85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.368646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-d091aea7-3a96-4f82-82ea-58eee232c62a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0bc9e010d044a5a2147010cbdb4d85'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.384534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1278/2578] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:09 2022] DELETE /identity/v3/users/ac0bc9e010d044a5a2147010cbdb4d85 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:09.572908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:09.585050 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa55cb-9463-4f67-9230-a5e2bc4d5c8b service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:09.685727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:09.686380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:09.686962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:09.686962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] PATH_INFO: `/v3/projects/b967c22662da47aeb9292704c39b44bb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:09.688314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b967c22662da47aeb9292704c39b44bb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:09.690372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:09.757254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b967c22662da47aeb9292704c39b44bb', 'request_id': 'req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.765839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b967c22662da47aeb9292704c39b44bb', 'request_id': 'req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.776507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b967c22662da47aeb9292704c39b44bb', 'request_id': 'req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.796946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] The token cache is being invalidate because project b967c22662da47aeb9292704c39b44bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:09.797272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.798121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d77a3db4-5f35-421b-96c0-0cf9e2f4e548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:09.800096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1301/2579] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:09 2022] DELETE /identity/v3/projects/b967c22662da47aeb9292704c39b44bb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:09.805545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d031-be51-4b4a-bdf4-53e6d0af14b9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:09.805833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d031-be51-4b4a-bdf4-53e6d0af14b9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:09.806119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d031-be51-4b4a-bdf4-53e6d0af14b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:09.829457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-bb25d031-be51-4b4a-bdf4-53e6d0af14b9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:09.834642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-bb25d031-be51-4b4a-bdf4-53e6d0af14b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:09.912589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1279/2580] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:09.920590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03426bf-c8af-4dba-8d84-239dfcb7f148 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:09.920954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03426bf-c8af-4dba-8d84-239dfcb7f148 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:09.921296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03426bf-c8af-4dba-8d84-239dfcb7f148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:09.949057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b03426bf-c8af-4dba-8d84-239dfcb7f148 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:09.950287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b03426bf-c8af-4dba-8d84-239dfcb7f148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:09.960135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1302/2581] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:09.964619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d031-be51-4b4a-bdf4-53e6d0af14b9 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:09.975324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d031-be51-4b4a-bdf4-53e6d0af14b9 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:09.980743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:09.981851 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:09.982306 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:09.982599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:09.983326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:09.983636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:09.985250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:09.998629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1280/2582] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:10.002850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03426bf-c8af-4dba-8d84-239dfcb7f148 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:10.007645 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:10.015202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03426bf-c8af-4dba-8d84-239dfcb7f148 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:10.022548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:10.023365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:10.023776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:10.024201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:10.024999 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:10.027179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:10.060098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1303/2583] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:10 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:15:10.064142 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:10.075416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b787af-aea7-4466-b95e-e9f074d186c2 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:10.093771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:10.139723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:10.140334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:10.140613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:10.140887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:10.141534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:10.143503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:10.166740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1281/2584] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:10 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:15:10.170941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:10.182347 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6926be-b9ad-4fe9-a11b-c8733ead9c1a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:10.187257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:10.187814 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:10.188095 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:10.188371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:10.189018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:10.191204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:10.201685 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1304/2585] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:10.206445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:10.216566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3ed75-52af-4bdb-a9d9-5f1b9c94be11 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:10.221536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236018a0-f966-4917-9e2f-1959449b7752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:10.222163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236018a0-f966-4917-9e2f-1959449b7752 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:10.222441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236018a0-f966-4917-9e2f-1959449b7752 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:10.222715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236018a0-f966-4917-9e2f-1959449b7752 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:10.223395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236018a0-f966-4917-9e2f-1959449b7752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:10.224996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236018a0-f966-4917-9e2f-1959449b7752 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:10.236235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1282/2586] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:10.240859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:10.250818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c9d75-1bcc-45ad-ac96-05fb378804cf admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:10.256257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:10.256932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:10.257238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:10.257512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] PATH_INFO: `/v3/projects/db7f044362844d7080d11f499151e113/users/4b22ae6fd0254a7295aa382046c81da4/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:10.274575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7f044362844d7080d11f499151e113, user_id=4b22ae6fd0254a7295aa382046c81da4, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:10.277245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:10.300039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1305/2587] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:10 2022] PUT /identity/v3/projects/db7f044362844d7080d11f499151e113/users/4b22ae6fd0254a7295aa382046c81da4/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:10.310155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a88c88-60f8-4891-b832-1cb220bf9890 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:10.310666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a88c88-60f8-4891-b832-1cb220bf9890 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:10.310752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a88c88-60f8-4891-b832-1cb220bf9890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:10.340827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a0a88c88-60f8-4891-b832-1cb220bf9890 None None] MFA Rules not processed for user `4b22ae6fd0254a7295aa382046c81da4`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:10.342064 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a0a88c88-60f8-4891-b832-1cb220bf9890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:10.408052 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1283/2588] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:10.419434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:10.421146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:10.480092 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb8990-aa85-4219-a699-f3e4ae7043e7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:10.484124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f3ad-4a3a-4831-b385-113ea7f958c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:10.484772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f3ad-4a3a-4831-b385-113ea7f958c7 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:10.485148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f3ad-4a3a-4831-b385-113ea7f958c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:10.485423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f3ad-4a3a-4831-b385-113ea7f958c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:10.494630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f3ad-4a3a-4831-b385-113ea7f958c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:10.496215 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f3ad-4a3a-4831-b385-113ea7f958c7 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:10.506402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1306/2589] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:19.001602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a88c88-60f8-4891-b832-1cb220bf9890 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:19.066663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a88c88-60f8-4891-b832-1cb220bf9890 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:19.071149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:19.071810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.072086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.072356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] PATH_INFO: `/v3/users/4b22ae6fd0254a7295aa382046c81da4` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.073463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b22ae6fd0254a7295aa382046c81da4)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:19.075455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:19.109463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:19.124301 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b22ae6fd0254a7295aa382046c81da4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.137156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b22ae6fd0254a7295aa382046c81da4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.137832 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b22ae6fd0254a7295aa382046c81da4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.138435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b22ae6fd0254a7295aa382046c81da4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.146013 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b22ae6fd0254a7295aa382046c81da4'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.158424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1284/2590] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:19 2022] DELETE /identity/v3/users/4b22ae6fd0254a7295aa382046c81da4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:19.162970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.163484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.164029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.183242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:19.184605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:19.251527 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1307/2591] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:19.261824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:19.263310 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:19.327536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24a93-f495-4f88-ad09-85d0e19e8494 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:19.331341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c03c8-5a12-4a30-b902-ff491fa66450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:19.331949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c03c8-5a12-4a30-b902-ff491fa66450 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.332263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c03c8-5a12-4a30-b902-ff491fa66450 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.332525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c03c8-5a12-4a30-b902-ff491fa66450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.341970 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c03c8-5a12-4a30-b902-ff491fa66450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:19.343436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c03c8-5a12-4a30-b902-ff491fa66450 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:19.353552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1285/2592] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:19.638668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:19.640143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:19.652354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0504-3a76-46a9-bf0f-141733bbd3d6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:19.657002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:19.657632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.657907 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.658177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] PATH_INFO: `/v3/projects/db7f044362844d7080d11f499151e113` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.659459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7f044362844d7080d11f499151e113)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:19.661434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:19.711172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:19.716024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7f044362844d7080d11f499151e113', 'request_id': 'req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.724022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7f044362844d7080d11f499151e113', 'request_id': 'req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.729754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7f044362844d7080d11f499151e113', 'request_id': 'req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.744221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] The token cache is being invalidate because project db7f044362844d7080d11f499151e113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:19.744512 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.745129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-dc7cd616-843e-4fbb-b0b1-9d2e8e683824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:19.746872 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1308/2593] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:19 2022] DELETE /identity/v3/projects/db7f044362844d7080d11f499151e113 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:19.751622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606b3e-3a6c-459f-9315-17dd748d25be None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.752032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606b3e-3a6c-459f-9315-17dd748d25be None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.752363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606b3e-3a6c-459f-9315-17dd748d25be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.773428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-8f606b3e-3a6c-459f-9315-17dd748d25be None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:19.774643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8f606b3e-3a6c-459f-9315-17dd748d25be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:19.858648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1286/2594] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:19.868282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b21f-088e-4885-bcdd-5b668301ca86 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.868571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b21f-088e-4885-bcdd-5b668301ca86 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.868851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b21f-088e-4885-bcdd-5b668301ca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.893988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7e60b21f-088e-4885-bcdd-5b668301ca86 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:19.895438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7e60b21f-088e-4885-bcdd-5b668301ca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:19.909563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1309/2595] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:19.910360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f606b3e-3a6c-459f-9315-17dd748d25be None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:19.920916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f606b3e-3a6c-459f-9315-17dd748d25be None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:19.926781 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:19.927440 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.927722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.927992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.928645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:19.929307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:19.930678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:19.947249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1287/2596] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:15:19.949043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b21f-088e-4885-bcdd-5b668301ca86 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:19.960720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b21f-088e-4885-bcdd-5b668301ca86 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:19.966548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:19.967179 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:19.967809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:19.968064 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:19.969259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:19.971992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:20.011402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1310/2597] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:19 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:20.013522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:20.026537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2202a6-48df-496b-8aaf-a5d0ddbee1a3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:20.067930 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:20.103229 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:20.103934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.104254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.104566 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.105234 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:20.107303 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:20.134097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1288/2598] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:20 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:20.138480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:20.159695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d538003-cc54-415e-82b2-283fac53ec3b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:20.159695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:20.159695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.159695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.160183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.160183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:20.162289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:20.177096 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1311/2599] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:20.181598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:20.199994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7a9a0-fe4f-418c-98e7-86cdb1ffee93 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:20.218225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1a2e2-6b54-487d-9555-6cfa98efc471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:20.219744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a2e2-6b54-487d-9555-6cfa98efc471 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.220220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a2e2-6b54-487d-9555-6cfa98efc471 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.220658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a2e2-6b54-487d-9555-6cfa98efc471 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.221702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1a2e2-6b54-487d-9555-6cfa98efc471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:20.223895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1a2e2-6b54-487d-9555-6cfa98efc471 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:20.242583 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1289/2600] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:20.247209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:20.258644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dedc3d-2202-4a8c-b75c-81512fad2a8b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:20.264515 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:20.265288 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.265555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.266031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] PATH_INFO: `/v3/projects/3ff89830f9b344129fe765c46da1ce88/users/e07e729f54ec45e4ad472fd1739a606d/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.288036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff89830f9b344129fe765c46da1ce88, user_id=e07e729f54ec45e4ad472fd1739a606d, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:20.291938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:20.317331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1312/2601] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:20 2022] PUT /identity/v3/projects/3ff89830f9b344129fe765c46da1ce88/users/e07e729f54ec45e4ad472fd1739a606d/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:15:20.320358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea009-9dc3-4023-bd2e-ba9e7c7e0315 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.320667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea009-9dc3-4023-bd2e-ba9e7c7e0315 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.320907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea009-9dc3-4023-bd2e-ba9e7c7e0315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.352946 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-84aea009-9dc3-4023-bd2e-ba9e7c7e0315 None None] MFA Rules not processed for user `e07e729f54ec45e4ad472fd1739a606d`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:20.354166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-84aea009-9dc3-4023-bd2e-ba9e7c7e0315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:20.438841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1290/2602] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:15:20.450827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:20.452659 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:20.535879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6c751-6f80-40ee-bdd7-bca72022820f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:20.541710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366825e-582f-4fc9-9997-2946ffd51c4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:20.542311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366825e-582f-4fc9-9997-2946ffd51c4e service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.542588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366825e-582f-4fc9-9997-2946ffd51c4e service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.542857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366825e-582f-4fc9-9997-2946ffd51c4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.552311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366825e-582f-4fc9-9997-2946ffd51c4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:20.553991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366825e-582f-4fc9-9997-2946ffd51c4e service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:20.563561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1313/2603] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:20.771017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea009-9dc3-4023-bd2e-ba9e7c7e0315 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:20.834876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea009-9dc3-4023-bd2e-ba9e7c7e0315 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:20.839418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:20.840053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.840332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.840609 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] PATH_INFO: `/v3/users/e07e729f54ec45e4ad472fd1739a606d` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.841725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07e729f54ec45e4ad472fd1739a606d)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:20.843678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:20.890773 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07e729f54ec45e4ad472fd1739a606d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:20.891968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:20.905587 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07e729f54ec45e4ad472fd1739a606d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:20.906262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07e729f54ec45e4ad472fd1739a606d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:20.906874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07e729f54ec45e4ad472fd1739a606d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:20.914279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07e729f54ec45e4ad472fd1739a606d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:20.929172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1291/2604] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:20 2022] DELETE /identity/v3/users/e07e729f54ec45e4ad472fd1739a606d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:20.933734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:20.934023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:20.934354 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:20.953813 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:20.955172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.001921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:21.017201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1314/2605] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:21.027235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.028647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.086445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf791be-8dff-4e84-a7f4-1f2785d720fb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.090259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ccb45-855a-4421-8ac2-05aaad383a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.090868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ccb45-855a-4421-8ac2-05aaad383a3b service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.091184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ccb45-855a-4421-8ac2-05aaad383a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.091459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ccb45-855a-4421-8ac2-05aaad383a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.101293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ccb45-855a-4421-8ac2-05aaad383a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.102714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ccb45-855a-4421-8ac2-05aaad383a3b service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.112312 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1292/2606] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:21.390049 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.392042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.404578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278dee-2dc5-4b85-94da-c67f64811241 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.410481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.411324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.411604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.411881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] PATH_INFO: `/v3/projects/3ff89830f9b344129fe765c46da1ce88` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.413056 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff89830f9b344129fe765c46da1ce88)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.415039 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.462972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff89830f9b344129fe765c46da1ce88', 'request_id': 'req-40247b63-4f04-4ce0-9d7f-c6914fe14c84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:21.469178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff89830f9b344129fe765c46da1ce88', 'request_id': 'req-40247b63-4f04-4ce0-9d7f-c6914fe14c84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:21.475430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff89830f9b344129fe765c46da1ce88', 'request_id': 'req-40247b63-4f04-4ce0-9d7f-c6914fe14c84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:21.490688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] The token cache is being invalidate because project 3ff89830f9b344129fe765c46da1ce88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:21.491178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:21.492045 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-40247b63-4f04-4ce0-9d7f-c6914fe14c84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:21.493824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1315/2607] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:21 2022] DELETE /identity/v3/projects/3ff89830f9b344129fe765c46da1ce88 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:21.498927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde037-0019-4f14-827a-c8533c6e31f9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.499245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde037-0019-4f14-827a-c8533c6e31f9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.499522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde037-0019-4f14-827a-c8533c6e31f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.523896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-81fde037-0019-4f14-827a-c8533c6e31f9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:21.525006 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-81fde037-0019-4f14-827a-c8533c6e31f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.592524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1293/2608] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:21.606591 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac04fb6-9570-413b-a3f6-6cdae437df53 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.606968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac04fb6-9570-413b-a3f6-6cdae437df53 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.607341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac04fb6-9570-413b-a3f6-6cdae437df53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.628832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-aac04fb6-9570-413b-a3f6-6cdae437df53 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:21.630024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-aac04fb6-9570-413b-a3f6-6cdae437df53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:21.637778 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1316/2609] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:15:21.641989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fde037-0019-4f14-827a-c8533c6e31f9 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.652328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fde037-0019-4f14-827a-c8533c6e31f9 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.657169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.657769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.658046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.658314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.659833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:21.660196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.661860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.673917 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1294/2610] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:15:21.678436 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac04fb6-9570-413b-a3f6-6cdae437df53 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.687867 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac04fb6-9570-413b-a3f6-6cdae437df53 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.692571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.693198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.693483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.693757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.694396 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.696346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.724005 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1317/2611] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:21 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:21.728245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.739396 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9d184-a466-45ba-9ece-5bbdf511c526 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.788699 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:21.796905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.797912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.798642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.798968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.805048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.805048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.825346 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1295/2612] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:21 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:21.829783 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.840517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ceaf36-0632-4f70-9872-2b2fe6c52faf admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.845365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.845977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.846448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.846919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.847785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.849409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.860424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1318/2613] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:21.864829 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.874735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6604df-5029-4328-b61b-938818a7f131 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.880712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8dd40-779b-4caa-8910-6205433aea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.881330 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dd40-779b-4caa-8910-6205433aea74 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.881607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dd40-779b-4caa-8910-6205433aea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.881876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8dd40-779b-4caa-8910-6205433aea74 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.882517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8dd40-779b-4caa-8910-6205433aea74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.884124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8dd40-779b-4caa-8910-6205433aea74 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.895135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1296/2614] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:21.899324 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:21.909331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beba547-ee67-4e35-90ed-a8b396d1d15a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:21.913899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:21.914540 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.914821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.915135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] PATH_INFO: `/v3/projects/d3a175f1ac2d4216a35fbaa72a683b8e/users/3909b610f14943eb90564d671ad5ca05/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.931018 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a175f1ac2d4216a35fbaa72a683b8e, user_id=3909b610f14943eb90564d671ad5ca05, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:21.933723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:21.947651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1319/2615] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:21 2022] PUT /identity/v3/projects/d3a175f1ac2d4216a35fbaa72a683b8e/users/3909b610f14943eb90564d671ad5ca05/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:21.959424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb204af-5ad0-4748-918e-727d46dd5150 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:21.959714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb204af-5ad0-4748-918e-727d46dd5150 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:21.959996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb204af-5ad0-4748-918e-727d46dd5150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:21.988939 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0fb204af-5ad0-4748-918e-727d46dd5150 None None] MFA Rules not processed for user `3909b610f14943eb90564d671ad5ca05`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:21.990173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0fb204af-5ad0-4748-918e-727d46dd5150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:22.054401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1297/2616] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:22.059171 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:22.120658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7feb83-f3a5-460b-aaeb-4e73b0137a3a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:22.125361 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:22.125952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.126229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.126514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.127152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:22.129548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:22.150336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:22.159304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1320/2617] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:22 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:15:22.162253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb204af-5ad0-4748-918e-727d46dd5150 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:22.172731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb204af-5ad0-4748-918e-727d46dd5150 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:22.232177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:22.232794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.233077 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.233368 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.233992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:22.235949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:22.259468 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1298/2618] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:22 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:22.263482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:22.274708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f3ee8-7761-41a5-8952-68f11d4d5f14 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:22.279380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:22.280013 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.280292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.280565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.281209 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:22.283162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:22.295914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1321/2619] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:22.301120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:22.311588 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79ebec-271f-4744-abca-3bb2632afbba admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:22.318704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:22.319423 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.319698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.319969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] PATH_INFO: `/v3/projects/7ef6dfd8a5fc450590b87b61b9748eb1/users/3227268b413748d28ddcd76361418625/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.335638 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef6dfd8a5fc450590b87b61b9748eb1, user_id=3227268b413748d28ddcd76361418625, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:22.338261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:22.352696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1299/2620] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:22 2022] PUT /identity/v3/projects/7ef6dfd8a5fc450590b87b61b9748eb1/users/3227268b413748d28ddcd76361418625/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:15:22.357029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:22.368331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b81c6b-129f-4a26-8968-8ada7d236a42 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:22.453297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e46e4-f68e-44ef-9b13-dda1a22fd62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:22.453969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e46e4-f68e-44ef-9b13-dda1a22fd62c admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.454274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e46e4-f68e-44ef-9b13-dda1a22fd62c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.455017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e46e4-f68e-44ef-9b13-dda1a22fd62c admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.455581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e46e4-f68e-44ef-9b13-dda1a22fd62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:22.457186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e46e4-f68e-44ef-9b13-dda1a22fd62c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:22.469040 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1322/2621] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:22.473714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:22.484307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7716bf5-1ba4-4f47-a844-ae4fd6102d34 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:22.489803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:22.490480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.490757 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.491030 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] PATH_INFO: `/v3/projects/7ef6dfd8a5fc450590b87b61b9748eb1/users/3227268b413748d28ddcd76361418625/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.492964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef6dfd8a5fc450590b87b61b9748eb1, user_id=3227268b413748d28ddcd76361418625, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:22.495614 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:22.506731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1300/2622] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:22 2022] PUT /identity/v3/projects/7ef6dfd8a5fc450590b87b61b9748eb1/users/3227268b413748d28ddcd76361418625/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:22.516844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72967-2611-48f6-95d1-723d22df8846 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.517137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72967-2611-48f6-95d1-723d22df8846 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.517415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72967-2611-48f6-95d1-723d22df8846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.546255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-7ee72967-2611-48f6-95d1-723d22df8846 None None] MFA Rules not processed for user `3227268b413748d28ddcd76361418625`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:22.547539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-7ee72967-2611-48f6-95d1-723d22df8846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:22.617178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1323/2623] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:22.630329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:22.632171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:22.651892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:22.693931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823835-434b-469c-b026-2e3a8200d839 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:22.698333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4501cdc-4eeb-4214-867a-c6b021f8deb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:22.698954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501cdc-4eeb-4214-867a-c6b021f8deb1 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:22.699276 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501cdc-4eeb-4214-867a-c6b021f8deb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:22.699554 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501cdc-4eeb-4214-867a-c6b021f8deb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:22.709097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4501cdc-4eeb-4214-867a-c6b021f8deb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:22.710536 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4501cdc-4eeb-4214-867a-c6b021f8deb1 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:22.720316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1301/2624] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:23.138656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee72967-2611-48f6-95d1-723d22df8846 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:23.201374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee72967-2611-48f6-95d1-723d22df8846 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:23.205961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:23.206574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:23.206854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:23.207153 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] PATH_INFO: `/v3/users/3909b610f14943eb90564d671ad5ca05` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:23.208291 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3909b610f14943eb90564d671ad5ca05)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:23.210203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:23.221119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:23.256951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3909b610f14943eb90564d671ad5ca05'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.264567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3909b610f14943eb90564d671ad5ca05'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.265460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3909b610f14943eb90564d671ad5ca05'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.278167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3909b610f14943eb90564d671ad5ca05'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.287413 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3909b610f14943eb90564d671ad5ca05'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.293949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1324/2625] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:23 2022] DELETE /identity/v3/users/3909b610f14943eb90564d671ad5ca05 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:23.298712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:23.299730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:23.300113 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:23.322771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:23.324017 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:23.386767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1302/2626] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:23.397287 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:23.398699 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:23.463343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0568a-182b-430c-bfeb-1bd3ce565663 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:23.467742 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:23.468357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:23.468635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:23.468907 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:23.480116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:23.481572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:23.493118 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1325/2627] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:23.773997 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:23.775482 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:23.792095 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938182b2-5916-47ef-90e5-3facc7c3105d None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:23.796891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:23.797539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:23.797902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:23.798108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] PATH_INFO: `/v3/projects/d3a175f1ac2d4216a35fbaa72a683b8e` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:23.800564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a175f1ac2d4216a35fbaa72a683b8e)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:23.802480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:23.862883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a175f1ac2d4216a35fbaa72a683b8e', 'request_id': 'req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.869165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a175f1ac2d4216a35fbaa72a683b8e', 'request_id': 'req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.877658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a175f1ac2d4216a35fbaa72a683b8e', 'request_id': 'req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.888945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] The token cache is being invalidate because project d3a175f1ac2d4216a35fbaa72a683b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:23.889160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.889779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:23.891320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1303/2628] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:23 2022] DELETE /identity/v3/projects/d3a175f1ac2d4216a35fbaa72a683b8e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:23.896334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:23.906854 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2afaf4-b5a2-40d9-a99f-150918acae53 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:23.967196 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:23.967827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:23.968107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:23.968379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] PATH_INFO: `/v3/users/3227268b413748d28ddcd76361418625` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:23.969502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3227268b413748d28ddcd76361418625)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:23.972212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:24.007367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:24.023372 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3227268b413748d28ddcd76361418625'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.032015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3227268b413748d28ddcd76361418625'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.032702 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3227268b413748d28ddcd76361418625'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.047201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3227268b413748d28ddcd76361418625'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.054822 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4dd6e06a-2a19-43eb-8180-3849c2a64855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3227268b413748d28ddcd76361418625'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.062477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1326/2629] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:23 2022] DELETE /identity/v3/users/3227268b413748d28ddcd76361418625 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:24.379890 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:24.382015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:24.418001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd87080-d50f-429d-b6b6-9b7cb8ab747b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:24.475307 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:24.504497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:24.505243 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:24.505525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:24.505796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] PATH_INFO: `/v3/projects/7ef6dfd8a5fc450590b87b61b9748eb1` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:24.507004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef6dfd8a5fc450590b87b61b9748eb1)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:24.508998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:24.568647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef6dfd8a5fc450590b87b61b9748eb1', 'request_id': 'req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.577491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef6dfd8a5fc450590b87b61b9748eb1', 'request_id': 'req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.585964 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef6dfd8a5fc450590b87b61b9748eb1', 'request_id': 'req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.599717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] The token cache is being invalidate because project 7ef6dfd8a5fc450590b87b61b9748eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:24.600021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.600697 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1cb02ff3-5f95-4d90-b0e1-2b9b1a92e4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:24.602261 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1304/2630] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:24 2022] DELETE /identity/v3/projects/7ef6dfd8a5fc450590b87b61b9748eb1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:24.607297 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c04023-5bff-4f9c-a533-b9418f51d66d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:24.607876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c04023-5bff-4f9c-a533-b9418f51d66d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:24.608026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c04023-5bff-4f9c-a533-b9418f51d66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:24.629886 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-d8c04023-5bff-4f9c-a533-b9418f51d66d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:24.631121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d8c04023-5bff-4f9c-a533-b9418f51d66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:24.702364 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1327/2631] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:24.711005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac6a85-3135-4b25-9aaf-8d7c06c30ecf None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:24.711567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac6a85-3135-4b25-9aaf-8d7c06c30ecf None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:24.711567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac6a85-3135-4b25-9aaf-8d7c06c30ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:24.732430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-84ac6a85-3135-4b25-9aaf-8d7c06c30ecf None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:24.734021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-84ac6a85-3135-4b25-9aaf-8d7c06c30ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:24.742491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1305/2632] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:24.746817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c04023-5bff-4f9c-a533-b9418f51d66d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:24.756864 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c04023-5bff-4f9c-a533-b9418f51d66d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:24.761555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:24.762293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:24.762575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:24.762849 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:24.763517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:24.763819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:24.765402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:24.778542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1328/2633] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:24.783325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac6a85-3135-4b25-9aaf-8d7c06c30ecf None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:24.793505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac6a85-3135-4b25-9aaf-8d7c06c30ecf None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:24.799569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:24.800189 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:24.800469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:24.800743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:24.801426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:24.803369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:24.837007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1306/2634] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:24 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:24.838552 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:24.849558 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5974a-10db-4025-889a-e325496deb17 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:24.919231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:24.920403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:24.920403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:24.920403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:24.920978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:24.922879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:24.953981 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1329/2635] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:24 2022] POST /identity/v3/users => generated 304 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:24.954921 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:24.969746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1f7e2-1f0c-4c34-9c0d-c4bdcedbbdf2 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:24.975547 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:24.976223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:24.976646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:24.977141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:24.978572 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:24.979902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:24.992085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1307/2636] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:24.996834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.011290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf75ff8-b607-48a4-8103-1de92be31be8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.016114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29047f9c-70c6-4029-a0f6-fd832ef20b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.016806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29047f9c-70c6-4029-a0f6-fd832ef20b02 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.017160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29047f9c-70c6-4029-a0f6-fd832ef20b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.017508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29047f9c-70c6-4029-a0f6-fd832ef20b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.018951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29047f9c-70c6-4029-a0f6-fd832ef20b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.020897 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29047f9c-70c6-4029-a0f6-fd832ef20b02 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.032173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1330/2637] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:25.036648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.047428 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a544f3-59d5-406b-9fdd-8e22c88dcd40 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.052662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.053425 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.053713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.054214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] PATH_INFO: `/v3/projects/09d33160804048ae8af31e83b259397e/users/3a4ae760c97d4786887c30ad8498049c/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.071190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d33160804048ae8af31e83b259397e, user_id=3a4ae760c97d4786887c30ad8498049c, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.073903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.089675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1308/2638] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:25 2022] PUT /identity/v3/projects/09d33160804048ae8af31e83b259397e/users/3a4ae760c97d4786887c30ad8498049c/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:25.101375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8e7cf-519d-492d-9a6d-1938698413b2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.101662 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8e7cf-519d-492d-9a6d-1938698413b2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.101938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8e7cf-519d-492d-9a6d-1938698413b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.130158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a4e8e7cf-519d-492d-9a6d-1938698413b2 None None] MFA Rules not processed for user `3a4ae760c97d4786887c30ad8498049c`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:25.131542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a4e8e7cf-519d-492d-9a6d-1938698413b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:25.135486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:25.200903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1331/2639] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:25 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:25.205357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.266481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6cad-f77b-4490-8cf9-740a2cee1fe1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.272722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.273333 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.273610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.273879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.274510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.276437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.294172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:25.309490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1309/2640] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:25 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:25.314057 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8e7cf-519d-492d-9a6d-1938698413b2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.325182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8e7cf-519d-492d-9a6d-1938698413b2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.382842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.383484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.383764 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.384036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.384680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.386601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.409438 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1332/2641] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:25 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:15:25.413401 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.423880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada472e4-0a03-49ec-8217-4c63ae80c7bb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.428692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.429291 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.429577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.429853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.430525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.432143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.442808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1310/2642] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:25.447214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.457780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34b8c6-21be-4e03-a2dd-21d0a6553934 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.462446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.463131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.463415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.463695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] PATH_INFO: `/v3/projects/630bac9fcf9d426188a9a0a4e30ec556/users/4c2a32787236450488c56e40dbb257d9/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.479953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=630bac9fcf9d426188a9a0a4e30ec556, user_id=4c2a32787236450488c56e40dbb257d9, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.482636 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.496479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1333/2643] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:25 2022] PUT /identity/v3/projects/630bac9fcf9d426188a9a0a4e30ec556/users/4c2a32787236450488c56e40dbb257d9/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:25.500476 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.511512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e15200-c43d-4f96-b276-32981acc47d6 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.575736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de0e34-f8bb-4ef9-99c7-537bdf2b0d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.576348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de0e34-f8bb-4ef9-99c7-537bdf2b0d73 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.576624 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de0e34-f8bb-4ef9-99c7-537bdf2b0d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.576900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de0e34-f8bb-4ef9-99c7-537bdf2b0d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.577628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de0e34-f8bb-4ef9-99c7-537bdf2b0d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.579302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de0e34-f8bb-4ef9-99c7-537bdf2b0d73 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.590164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1311/2644] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:25.594387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.606393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa31f1c-a051-4995-a2e2-33553322be8e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.611302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.612631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.612920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.613197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] PATH_INFO: `/v3/projects/630bac9fcf9d426188a9a0a4e30ec556/users/4c2a32787236450488c56e40dbb257d9/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.615207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630bac9fcf9d426188a9a0a4e30ec556, user_id=4c2a32787236450488c56e40dbb257d9, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.617859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.629779 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1334/2645] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:25 2022] PUT /identity/v3/projects/630bac9fcf9d426188a9a0a4e30ec556/users/4c2a32787236450488c56e40dbb257d9/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:25.645155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9d72-5116-4247-90fe-39426f5402cb None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.645687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9d72-5116-4247-90fe-39426f5402cb None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.645961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d9d72-5116-4247-90fe-39426f5402cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.679999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-859d9d72-5116-4247-90fe-39426f5402cb None None] MFA Rules not processed for user `4c2a32787236450488c56e40dbb257d9`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:25.681459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-859d9d72-5116-4247-90fe-39426f5402cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:25.770961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1312/2646] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:25.780253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:25.781989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:25.846982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbcbc2-ad4a-4636-ab25-5ec93ad507a0 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:25.851473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc50a1-5261-4dc9-80c5-879121c56e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:25.852154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc50a1-5261-4dc9-80c5-879121c56e29 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:25.852506 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc50a1-5261-4dc9-80c5-879121c56e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:25.852846 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc50a1-5261-4dc9-80c5-879121c56e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:25.863423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc50a1-5261-4dc9-80c5-879121c56e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:25.864870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc50a1-5261-4dc9-80c5-879121c56e29 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:25.870121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:25.876100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1335/2647] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:25 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:26.028201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9d72-5116-4247-90fe-39426f5402cb None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:26.038316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d9d72-5116-4247-90fe-39426f5402cb None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:26.041916 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:26.042582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:26.042868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:26.043173 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:26.101260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:26.102705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:26.111951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1313/2648] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:26 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:29.069859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:29.070395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:29.070891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:29.093369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:29.094698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:29.152206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1336/2649] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:15:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:29.175611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:29.177102 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:29.182430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:29.259802 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ce88e-7a2f-494a-803a-bcbe74c058e5 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:29.265873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:29.266593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:29.266971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:29.267336 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:29.276677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:29.278299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:29.288453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1314/2650] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:15:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:15:39.013074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:39.015714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:39.126315 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ee76-88a7-4316-adc0-57228e8b48f8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:39.131394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:39.132164 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:39.132691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:39.132961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:39.135178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:39.218719 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:39.220495 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:39.243016 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1337/2651] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:15:39.283128 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=31 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:39.878866 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:39.882772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:39.895114 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f4285-5510-4ade-aafd-dc2cf8b4c228 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:39.900063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad6859-1e26-4986-a7fa-e9e8c98f603e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:39.900670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6859-1e26-4986-a7fa-e9e8c98f603e service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:39.900946 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6859-1e26-4986-a7fa-e9e8c98f603e service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:39.901482 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad6859-1e26-4986-a7fa-e9e8c98f603e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:39.903817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-16ad6859-1e26-4986-a7fa-e9e8c98f603e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:39.927435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:40.005030 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad6859-1e26-4986-a7fa-e9e8c98f603e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:40.006954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad6859-1e26-4986-a7fa-e9e8c98f603e service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:40.031256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1338/2652] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:15:40.107409 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30ce13-b97e-43a7-a2d0-cb6d1c9eaa31 service swift] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:40.114684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:40.115959 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:40.116263 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:40.116580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] PATH_INFO: `/v3/users/3a4ae760c97d4786887c30ad8498049c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:40.118115 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a4ae760c97d4786887c30ad8498049c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:40.122895 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:40.216990 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a4ae760c97d4786887c30ad8498049c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:40.262887 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a4ae760c97d4786887c30ad8498049c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:40.263685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a4ae760c97d4786887c30ad8498049c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:40.264328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a4ae760c97d4786887c30ad8498049c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:40.281224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a4ae760c97d4786887c30ad8498049c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:40.301960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1315/2653] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:39 2022] DELETE /identity/v3/users/3a4ae760c97d4786887c30ad8498049c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:40.307432 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:40.307663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:40.307913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:40.368562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:40.370128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:40.459219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1339/2654] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:40.470202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:40.473088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:40.580085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:40.619564 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7e920-1600-45cd-bf83-6f58b685022b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:40.626223 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:40.626872 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:40.627178 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:40.627617 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:40.647201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:40.650775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:40.667580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1316/2655] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:41.123004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:41.124682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:41.141762 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f1382-f213-4ad1-a940-41e5acf82f2e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:41.148453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:41.149219 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:41.149584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:41.149926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] PATH_INFO: `/v3/projects/09d33160804048ae8af31e83b259397e` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:41.151172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d33160804048ae8af31e83b259397e)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:41.153203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:41.206717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d33160804048ae8af31e83b259397e', 'request_id': 'req-d38b5c91-5808-4279-9b9c-298205c67784'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.213265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d33160804048ae8af31e83b259397e', 'request_id': 'req-d38b5c91-5808-4279-9b9c-298205c67784'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.219544 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d33160804048ae8af31e83b259397e', 'request_id': 'req-d38b5c91-5808-4279-9b9c-298205c67784'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.233994 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] The token cache is being invalidate because project 09d33160804048ae8af31e83b259397e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:41.234797 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.235468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.236952 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1340/2656] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:41 2022] DELETE /identity/v3/projects/09d33160804048ae8af31e83b259397e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:41.241667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:41.251968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd978836-3a3d-443d-bc96-cf692a59a087 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:41.334141 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:41.334811 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:41.335047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:41.335352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] PATH_INFO: `/v3/users/4c2a32787236450488c56e40dbb257d9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:41.336474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2a32787236450488c56e40dbb257d9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:41.338379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:41.389948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2a32787236450488c56e40dbb257d9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.406309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2a32787236450488c56e40dbb257d9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.407045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2a32787236450488c56e40dbb257d9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.407782 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2a32787236450488c56e40dbb257d9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.415922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-6c470593-8032-48da-a143-e01bc7111696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2a32787236450488c56e40dbb257d9'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.433383 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1317/2657] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:41 2022] DELETE /identity/v3/users/4c2a32787236450488c56e40dbb257d9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:41.717378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:41.718905 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:41.732834 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5c91-5808-4279-9b9c-298205c67784 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:41.751550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:41.797140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:41.797998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:41.798283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:41.798555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] PATH_INFO: `/v3/projects/630bac9fcf9d426188a9a0a4e30ec556` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:41.805467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=630bac9fcf9d426188a9a0a4e30ec556)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:41.805467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:41.859387 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630bac9fcf9d426188a9a0a4e30ec556', 'request_id': 'req-0703ced4-9e0b-475a-8d2c-b4860195b9f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.866319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630bac9fcf9d426188a9a0a4e30ec556', 'request_id': 'req-0703ced4-9e0b-475a-8d2c-b4860195b9f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.872083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630bac9fcf9d426188a9a0a4e30ec556', 'request_id': 'req-0703ced4-9e0b-475a-8d2c-b4860195b9f2'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.884891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] The token cache is being invalidate because project 630bac9fcf9d426188a9a0a4e30ec556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:41.885180 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.885808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-0703ced4-9e0b-475a-8d2c-b4860195b9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:41.887397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1341/2658] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:41 2022] DELETE /identity/v3/projects/630bac9fcf9d426188a9a0a4e30ec556 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:41.892318 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8fe3a-a670-4acc-b231-8c125e957f65 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:41.892612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8fe3a-a670-4acc-b231-8c125e957f65 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:41.892888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8fe3a-a670-4acc-b231-8c125e957f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:41.942828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f6c8fe3a-a670-4acc-b231-8c125e957f65 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:41.944082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f6c8fe3a-a670-4acc-b231-8c125e957f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:41.974690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:42.007751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1318/2659] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:15:42.016857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a9d9-5b71-4900-8bdf-21bd5273cb3e None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.017143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a9d9-5b71-4900-8bdf-21bd5273cb3e None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.017419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a9d9-5b71-4900-8bdf-21bd5273cb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.037415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8fe4a9d9-5b71-4900-8bdf-21bd5273cb3e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:42.038621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8fe4a9d9-5b71-4900-8bdf-21bd5273cb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:42.046535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1342/2660] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:42.052054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8fe3a-a670-4acc-b231-8c125e957f65 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:42.068161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8fe3a-a670-4acc-b231-8c125e957f65 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:42.079403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:42.080123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.080407 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.080681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.082510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:42.082819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:42.084533 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:42.097795 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1319/2661] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:42.102292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4a9d9-5b71-4900-8bdf-21bd5273cb3e None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:42.111691 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4a9d9-5b71-4900-8bdf-21bd5273cb3e None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:42.116477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:42.117078 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.117358 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.117630 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.118282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:42.120302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:42.148771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1343/2662] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:42 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:42.153319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:42.163950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa15655-9884-45b4-919a-a2ab8bfae6cf admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:42.222149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:42.222745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.223024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.223354 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.223996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:42.225907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:42.247193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1320/2663] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:42 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:42.251437 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:42.262283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6669-8792-49e8-a46b-42ef96b859a2 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:42.267116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:42.267723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.268022 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.268293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.269977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:42.271266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:42.281284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1344/2664] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:42.285271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:42.295758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f386d3-51f1-4e31-9d0e-718398986848 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:42.300944 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693e89-6a85-4f69-b645-b67007c36b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:42.301551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693e89-6a85-4f69-b645-b67007c36b03 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.301834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693e89-6a85-4f69-b645-b67007c36b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.302109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693e89-6a85-4f69-b645-b67007c36b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.302761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54693e89-6a85-4f69-b645-b67007c36b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:42.304391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54693e89-6a85-4f69-b645-b67007c36b03 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:42.315054 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1321/2665] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:15:42.319943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:42.330259 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d971f-82ff-49d6-9931-06664a2f6b7b admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:42.335706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:42.336359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.336639 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.336943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] PATH_INFO: `/v3/projects/e1c15ff898954850a76318cc9209a9b4/users/9d4186e846f7462dbbb27bbf1f84e33f/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.353648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c15ff898954850a76318cc9209a9b4, user_id=9d4186e846f7462dbbb27bbf1f84e33f, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:42.356375 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:42.370076 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1345/2666] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:42 2022] PUT /identity/v3/projects/e1c15ff898954850a76318cc9209a9b4/users/9d4186e846f7462dbbb27bbf1f84e33f/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:42.381416 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.381998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.381998 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.413472 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] MFA Rules not processed for user `9d4186e846f7462dbbb27bbf1f84e33f`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:42.414713 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:42.484808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1322/2667] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:42.496619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:42.498289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:42.552199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:42.556217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a8daa-b8ce-4890-9353-7c47e8582ae9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:42.561126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92623-b9b0-4642-9eeb-5a33848a33ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:42.561856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92623-b9b0-4642-9eeb-5a33848a33ce service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:42.562501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92623-b9b0-4642-9eeb-5a33848a33ce service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:42.562819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92623-b9b0-4642-9eeb-5a33848a33ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:42.572501 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd92623-b9b0-4642-9eeb-5a33848a33ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:42.573956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd92623-b9b0-4642-9eeb-5a33848a33ce service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:42.582726 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1346/2668] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:43.904436 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:43.906520 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:43.947788 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:43.989493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6a376-dac0-4a52-a260-fab877055784 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:43.995148 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:43.995842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:43.996127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:43.996430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] PATH_INFO: `/v3/users/30cd65c8b64d4a6e955d678096e4e927` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:43.997907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cd65c8b64d4a6e955d678096e4e927)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:44.000020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:44.062098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cd65c8b64d4a6e955d678096e4e927'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:44.084152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cd65c8b64d4a6e955d678096e4e927'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:44.085460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cd65c8b64d4a6e955d678096e4e927'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:44.086818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cd65c8b64d4a6e955d678096e4e927'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:44.101021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cd65c8b64d4a6e955d678096e4e927'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:44.125494 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1323/2669] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:15:43 2022] DELETE /identity/v3/users/30cd65c8b64d4a6e955d678096e4e927 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:15:44.130948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:44.131504 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:44.131618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:44.151953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:44.153236 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:44.222595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1347/2670] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:44.232776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:44.234286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:44.304537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91622adb-3b36-4b37-a242-5f01fcc05526 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:44.308451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2da40-058f-4964-86dd-37014f4bae8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:44.309144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2da40-058f-4964-86dd-37014f4bae8f service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:44.309549 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2da40-058f-4964-86dd-37014f4bae8f service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:44.309931 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2da40-058f-4964-86dd-37014f4bae8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:44.320198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2da40-058f-4964-86dd-37014f4bae8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:44.321767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2da40-058f-4964-86dd-37014f4bae8f service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:44.333192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1324/2671] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:44.826169 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:44.828130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:44.876583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8975047-a47e-4307-9619-845d3d7a4294 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:44.883964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:44.884608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:44.884915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:44.885539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] PATH_INFO: `/v3/projects/0c47352146d540369a0b4079a5ba9e8f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:44.887331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c47352146d540369a0b4079a5ba9e8f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:44.890594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:45.035477 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c47352146d540369a0b4079a5ba9e8f', 'request_id': 'req-d3499015-3853-4c49-a0bf-fb9e9189ae45'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:45.045643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c47352146d540369a0b4079a5ba9e8f', 'request_id': 'req-d3499015-3853-4c49-a0bf-fb9e9189ae45'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:45.068272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c47352146d540369a0b4079a5ba9e8f', 'request_id': 'req-d3499015-3853-4c49-a0bf-fb9e9189ae45'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:45.085264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] The token cache is being invalidate because project 0c47352146d540369a0b4079a5ba9e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:15:45.085663 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:45.086535 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d3499015-3853-4c49-a0bf-fb9e9189ae45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:15:45.088480 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1348/2672] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:15:44 2022] DELETE /identity/v3/projects/0c47352146d540369a0b4079a5ba9e8f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:15:45.093775 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39a214-f60c-4400-89da-ee73c4b579c6 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.094360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39a214-f60c-4400-89da-ee73c4b579c6 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.094868 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39a214-f60c-4400-89da-ee73c4b579c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.146647 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-4c39a214-f60c-4400-89da-ee73c4b579c6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:45.148218 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4c39a214-f60c-4400-89da-ee73c4b579c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:45.235392 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:45.244902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1325/2673] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:45.258017 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77f730-d0fe-4449-9c06-728083427339 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.258755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77f730-d0fe-4449-9c06-728083427339 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.259223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77f730-d0fe-4449-9c06-728083427339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.290337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4d77f730-d0fe-4449-9c06-728083427339 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:45.291825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4d77f730-d0fe-4449-9c06-728083427339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:45.307972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39a214-f60c-4400-89da-ee73c4b579c6 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:45.311154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1349/2674] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:45.319325 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39a214-f60c-4400-89da-ee73c4b579c6 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:45.324016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:45.325200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.325200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.325200 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.325818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:15:45.326349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:45.328069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:45.348806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1326/2675] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:15:45.350038 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77f730-d0fe-4449-9c06-728083427339 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:45.360703 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77f730-d0fe-4449-9c06-728083427339 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:45.367254 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:45.368167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.368677 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.369212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.370549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:45.373217 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:45.417334 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1350/2676] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:15:45 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:45.421681 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:45.438446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75142ed2-a9e8-4191-beaf-82f18051a2db admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:45.526677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:45.527756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.528212 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.528661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.529619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:45.531900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:45.567582 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1327/2677] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:15:45 2022] POST /identity/v3/users => generated 283 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:15:45.573258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:45.603692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6cbf6-0bf2-40f6-a484-d7a6543a63b3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:45.611015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:45.613012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.613548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.614011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.615270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:45.631570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:45.649251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1351/2678] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:45.649816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:45.666004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb6893-276a-483c-97b8-83384d3b0c40 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:45.677632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6c133-ceb2-4293-9875-79bb20fb221c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:45.678049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6c133-ceb2-4293-9875-79bb20fb221c admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.679833 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6c133-ceb2-4293-9875-79bb20fb221c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.680577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6c133-ceb2-4293-9875-79bb20fb221c admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.681531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6c133-ceb2-4293-9875-79bb20fb221c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:45.683901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6c133-ceb2-4293-9875-79bb20fb221c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:45.703762 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1328/2679] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:15:45.708253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:45.720747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fc505-3e4d-4f88-ae82-bd0fb928b9b4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:45.726500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:45.727366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.728089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.728249 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] PATH_INFO: `/v3/projects/402f1e46cc964d499eb67f83ed314f4b/users/274e57f093d24099831bd536e7b3a02c/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.748542 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=402f1e46cc964d499eb67f83ed314f4b, user_id=274e57f093d24099831bd536e7b3a02c, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:45.751371 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:45.769969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1352/2680] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:15:45 2022] PUT /identity/v3/projects/402f1e46cc964d499eb67f83ed314f4b/users/274e57f093d24099831bd536e7b3a02c/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:15:45.781055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ad736-1bae-4927-9183-fd1173dbe55c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:45.782714 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ad736-1bae-4927-9183-fd1173dbe55c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:45.783815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ad736-1bae-4927-9183-fd1173dbe55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:45.830539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-a45ad736-1bae-4927-9183-fd1173dbe55c None None] MFA Rules not processed for user `274e57f093d24099831bd536e7b3a02c`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:45.831742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a45ad736-1bae-4927-9183-fd1173dbe55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:45.955590 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1329/2681] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:45.964129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:45.966574 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:45.986028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:46.056178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf259-3d99-4016-9120-70ee3243d755 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:46.060132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:46.060617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:46.060969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:46.061252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:46.072290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:46.073759 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:46.087193 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1353/2682] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:15:48.370994 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d134930-57f9-463c-af85-78128e4bb823 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:48.371603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d134930-57f9-463c-af85-78128e4bb823 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:48.371603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d134930-57f9-463c-af85-78128e4bb823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:48.399272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-4d134930-57f9-463c-af85-78128e4bb823 None None] MFA Rules not processed for user `8f1c0b2189a041fdb8dbe2a0290ffac8`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:48.401002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-4d134930-57f9-463c-af85-78128e4bb823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:48.474445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:48.487551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1330/2683] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:15:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:51.136167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:51.144977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:51.243670 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8d482-84b7-4373-8b83-eca395b2a1c0 service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:51.247844 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:51.249295 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:51.249514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:51.250265 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:51.253682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:51.372737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:51.374881 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:51.396232 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1354/2684] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:15:55.010283 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:55.010815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:55.010815 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:55.062061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:15:55.075160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:55.211855 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1331/2685] 10.222.0.94 () {58 vars in 967 bytes} [Mon Aug 29 19:15:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:15:55.229607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:15:55.232230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:15:55.395216 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2d063-1d9c-444d-8a67-9e4e9ad470e5 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:15:55.407097 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:15:55.407725 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:15:55.408074 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:15:55.408272 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:15:55.431892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:15:55.437233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:15:55.438776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:15:55.479557 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1355/2686] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:15:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:16:01.258232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:01.259841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:01.283842 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5cc71-c4a8-4543-b8ca-a8cf946ad6d9 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:01.289351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:01.290085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:01.290390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:01.291003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] PATH_INFO: `/v3/users/82fb5d585b5d460bb258f929057a4093` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:01.292557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82fb5d585b5d460bb258f929057a4093)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:01.295140 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:01.346481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82fb5d585b5d460bb258f929057a4093'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.358847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82fb5d585b5d460bb258f929057a4093'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.359603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82fb5d585b5d460bb258f929057a4093'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.360327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82fb5d585b5d460bb258f929057a4093'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.367878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82fb5d585b5d460bb258f929057a4093'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.382947 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1332/2687] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:01 2022] DELETE /identity/v3/users/82fb5d585b5d460bb258f929057a4093 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:01.629239 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:01.647167 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75920e08-781c-4b63-bf5a-0cc342b697b5 service swift] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:01.726498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:01.727509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:01.727978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:01.728299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] PATH_INFO: `/v3/projects/f0f4bcf841c942a9928f5b343b5fb8bb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:01.729479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f4bcf841c942a9928f5b343b5fb8bb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:01.731442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:01.793915 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f4bcf841c942a9928f5b343b5fb8bb', 'request_id': 'req-5684d7aa-b5a0-4f03-b4af-dab438e3978e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.807818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f4bcf841c942a9928f5b343b5fb8bb', 'request_id': 'req-5684d7aa-b5a0-4f03-b4af-dab438e3978e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.814672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f4bcf841c942a9928f5b343b5fb8bb', 'request_id': 'req-5684d7aa-b5a0-4f03-b4af-dab438e3978e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.828317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] The token cache is being invalidate because project f0f4bcf841c942a9928f5b343b5fb8bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:01.828708 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.829252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.836369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1356/2688] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:01 2022] DELETE /identity/v3/projects/f0f4bcf841c942a9928f5b343b5fb8bb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:01.836556 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:01.837145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:01.851967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eee60d-8114-4e2f-b3ce-0c8a0671b7c8 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:01.906793 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:01.916672 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:01.917595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:01.917879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:01.918300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] PATH_INFO: `/v3/users/aeac5df40d57465e820d3b961633c5ac` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:01.919571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeac5df40d57465e820d3b961633c5ac)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:01.921550 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:01.974072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeac5df40d57465e820d3b961633c5ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.988574 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeac5df40d57465e820d3b961633c5ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.989441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeac5df40d57465e820d3b961633c5ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.990047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeac5df40d57465e820d3b961633c5ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:01.997766 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeac5df40d57465e820d3b961633c5ac'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.014988 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1333/2689] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:01 2022] DELETE /identity/v3/users/aeac5df40d57465e820d3b961633c5ac => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:02.147853 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:02.149609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:02.171709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684d7aa-b5a0-4f03-b4af-dab438e3978e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:02.266058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:02.266058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:02.266404 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:02.266631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] PATH_INFO: `/v3/users/bccc502170e349e0848922712c574ca7` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:02.267821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccc502170e349e0848922712c574ca7)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:02.270186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:02.311607 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:02.322405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:02.323918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:02.342910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccc502170e349e0848922712c574ca7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.346267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676642b9-763e-41d8-b8e7-4fe2d8d385a3 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:02.354694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:02.355695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:02.356460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:02.356460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] PATH_INFO: `/v3/projects/edf46f0b532246c5ad132be112175ca9` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:02.358749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf46f0b532246c5ad132be112175ca9)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:02.359541 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccc502170e349e0848922712c574ca7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.360618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccc502170e349e0848922712c574ca7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.368183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:02.390077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccc502170e349e0848922712c574ca7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.401309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccc502170e349e0848922712c574ca7'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.409871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1357/2690] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:02 2022] DELETE /identity/v3/users/bccc502170e349e0848922712c574ca7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:02.438228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf46f0b532246c5ad132be112175ca9', 'request_id': 'req-072bcbdf-9992-4d74-9217-687d58b42167'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.446808 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf46f0b532246c5ad132be112175ca9', 'request_id': 'req-072bcbdf-9992-4d74-9217-687d58b42167'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.456166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf46f0b532246c5ad132be112175ca9', 'request_id': 'req-072bcbdf-9992-4d74-9217-687d58b42167'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.482238 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] The token cache is being invalidate because project edf46f0b532246c5ad132be112175ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:02.482570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.484585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.487136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1334/2691] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:02 2022] DELETE /identity/v3/projects/edf46f0b532246c5ad132be112175ca9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:02.645554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:02.649089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:02.783172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cde783-ba8c-4182-85cb-3a25c9a919c9 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:02.860242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:02.861634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:02.865980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:02.866622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:02.866902 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:02.867213 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] PATH_INFO: `/v3/projects/abcda882db7046eab245492bc4c4d7e6` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:02.868379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abcda882db7046eab245492bc4c4d7e6)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:02.870357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:02.924969 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abcda882db7046eab245492bc4c4d7e6', 'request_id': 'req-62e3d856-4955-4ea3-8fff-2368566f97b4'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.927971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:02.932972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abcda882db7046eab245492bc4c4d7e6', 'request_id': 'req-62e3d856-4955-4ea3-8fff-2368566f97b4'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.933446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:02.941379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abcda882db7046eab245492bc4c4d7e6', 'request_id': 'req-62e3d856-4955-4ea3-8fff-2368566f97b4'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.951213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bcbdf-9992-4d74-9217-687d58b42167 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:02.955692 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] The token cache is being invalidate because project abcda882db7046eab245492bc4c4d7e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:02.955988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.957146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:02.959139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1358/2692] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:02 2022] DELETE /identity/v3/projects/abcda882db7046eab245492bc4c4d7e6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:03.010175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:03.010738 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:03.011024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:03.011335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:03.013270 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:03.087688 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:03.089162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:03.101237 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1335/2693] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:06.699407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:06.701242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:06.724376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3d856-4955-4ea3-8fff-2368566f97b4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:06.729428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:06.730089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:06.730369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:06.730640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] PATH_INFO: `/v3/users/ff8ce50a95b742a48500e634db50fb36` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:06.731898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8ce50a95b742a48500e634db50fb36)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:06.733910 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:06.806462 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8ce50a95b742a48500e634db50fb36'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:06.815004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8ce50a95b742a48500e634db50fb36'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:06.816592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8ce50a95b742a48500e634db50fb36'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:06.830183 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8ce50a95b742a48500e634db50fb36'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:06.842004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f5b7777e-05ad-4a64-9dcd-72fe3a73543b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8ce50a95b742a48500e634db50fb36'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:06.848772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1359/2694] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:06 2022] DELETE /identity/v3/users/ff8ce50a95b742a48500e634db50fb36 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:07.034137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.046433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5419-fe91-4869-b5f6-069b73c038fa service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.139398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.140483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.141048 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.141515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] PATH_INFO: `/v3/projects/6f6085f7a1df49ae8e2375e28d7d6792` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.143098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6085f7a1df49ae8e2375e28d7d6792)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.145819 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.203526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:07.208413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6085f7a1df49ae8e2375e28d7d6792', 'request_id': 'req-3f27e776-f159-4277-9007-863a59f00df1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:07.216001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6085f7a1df49ae8e2375e28d7d6792', 'request_id': 'req-3f27e776-f159-4277-9007-863a59f00df1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:07.224894 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6085f7a1df49ae8e2375e28d7d6792', 'request_id': 'req-3f27e776-f159-4277-9007-863a59f00df1'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:07.235922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] The token cache is being invalidate because project 6f6085f7a1df49ae8e2375e28d7d6792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:07.236220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:07.236877 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-3f27e776-f159-4277-9007-863a59f00df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:07.238497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1336/2695] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:07 2022] DELETE /identity/v3/projects/6f6085f7a1df49ae8e2375e28d7d6792 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:07.243454 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c5b0f-f15f-4420-a3b6-be9b4fbe6b33 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.243739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c5b0f-f15f-4420-a3b6-be9b4fbe6b33 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.244036 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c5b0f-f15f-4420-a3b6-be9b4fbe6b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.265326 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e90c5b0f-f15f-4420-a3b6-be9b4fbe6b33 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:07.266536 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e90c5b0f-f15f-4420-a3b6-be9b4fbe6b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:07.337513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1360/2696] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:07.347496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445680f-b310-420d-ab4c-49a142ae65bd None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.347780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445680f-b310-420d-ab4c-49a142ae65bd None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.348060 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445680f-b310-420d-ab4c-49a142ae65bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.370962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c445680f-b310-420d-ab4c-49a142ae65bd None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:07.372206 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c445680f-b310-420d-ab4c-49a142ae65bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:07.381398 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1337/2697] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:07.385652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c5b0f-f15f-4420-a3b6-be9b4fbe6b33 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.395738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c5b0f-f15f-4420-a3b6-be9b4fbe6b33 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.400460 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.401073 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.401366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.401638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.402290 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:07.402626 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.405266 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.418365 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1361/2698] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:07.427363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445680f-b310-420d-ab4c-49a142ae65bd None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.437983 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445680f-b310-420d-ab4c-49a142ae65bd None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.443519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.444133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.444415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.444695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.445347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.447281 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.482986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1338/2699] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:07 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:07.484459 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.495323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a66ed-6faa-4bbe-a749-d87181604d59 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.536735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:07.560753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.561417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.561706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.562024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.562683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.565208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.589553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1362/2700] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:07 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:07.594797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.605859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a456b6-18ec-4398-b4c0-64ac24e5c44c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.610961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.611598 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.611881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.612156 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.613324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.615000 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.626584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1339/2701] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:07.631609 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.644521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9d9a-026a-4546-87f2-05741e5969a4 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.649780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca6e03-f426-4be0-9e95-6cb7cbe4e0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.650448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6e03-f426-4be0-9e95-6cb7cbe4e0cc admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.650786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6e03-f426-4be0-9e95-6cb7cbe4e0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.651063 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca6e03-f426-4be0-9e95-6cb7cbe4e0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.651754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca6e03-f426-4be0-9e95-6cb7cbe4e0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.653386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca6e03-f426-4be0-9e95-6cb7cbe4e0cc admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.664950 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1363/2702] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:07.669061 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.681097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36efb49-d22a-4c12-97c8-677b99368297 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.690744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.692675 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.692960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.694198 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] PATH_INFO: `/v3/projects/ee503e1827984f328f02cf1a583ca4a2/users/e1d2030cb2f340f2b3f839befd5f8d16/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.711756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee503e1827984f328f02cf1a583ca4a2, user_id=e1d2030cb2f340f2b3f839befd5f8d16, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.714513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.735123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1340/2703] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:07 2022] PUT /identity/v3/projects/ee503e1827984f328f02cf1a583ca4a2/users/e1d2030cb2f340f2b3f839befd5f8d16/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:07.747248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb873118-15d3-4968-99fe-c81b00783a28 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.747727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb873118-15d3-4968-99fe-c81b00783a28 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.747815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb873118-15d3-4968-99fe-c81b00783a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.779244 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-cb873118-15d3-4968-99fe-c81b00783a28 None None] MFA Rules not processed for user `e1d2030cb2f340f2b3f839befd5f8d16`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:07.780478 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cb873118-15d3-4968-99fe-c81b00783a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:07.849695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1364/2704] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:07.854161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.918124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc79a95-273a-4aef-a0e7-a1318094ed31 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:07.922820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:07.923460 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:07.923744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:07.924018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:07.924702 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:07.927560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:07.955700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1341/2705] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:07 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:07.961116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb873118-15d3-4968-99fe-c81b00783a28 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:07.972136 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb873118-15d3-4968-99fe-c81b00783a28 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.033283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.033933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.034303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.034798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.035490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.037424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.061085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1365/2706] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:07 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:08.065630 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.077923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d61b69-f0f4-46e2-ba19-67cb826afa4a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.083040 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.083684 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.083962 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.084235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.084881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.086493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.097589 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1342/2707] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:08.101930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.112275 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63ad73-a7bb-42e8-a03e-3412225f5ace admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.118505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.119223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.119508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.119782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] PATH_INFO: `/v3/projects/27d1ad0a68e44a67b132601065b89dd6/users/6f789b1d3332413696fabc10a8531881/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.136048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d1ad0a68e44a67b132601065b89dd6, user_id=6f789b1d3332413696fabc10a8531881, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.138721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.151277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1366/2708] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:08 2022] PUT /identity/v3/projects/27d1ad0a68e44a67b132601065b89dd6/users/6f789b1d3332413696fabc10a8531881/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:08.155427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.166319 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cc2f5-d596-41bc-820c-fadce0e96ef7 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.209883 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:08.228331 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.228943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.229225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.229490 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.230294 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.231932 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.245756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1343/2709] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:08.245936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.245936 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:08.246686 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.257434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481774d-ad39-419a-b05b-26685b6579b4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.258728 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11754d5-fc19-43bc-a96d-1bbe5cb07ae2 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.262668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597474f-042a-4e42-82c9-c454b7c7f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.263481 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597474f-042a-4e42-82c9-c454b7c7f268 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.263768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597474f-042a-4e42-82c9-c454b7c7f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.264046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597474f-042a-4e42-82c9-c454b7c7f268 admin admin] PATH_INFO: `/v3/projects/27d1ad0a68e44a67b132601065b89dd6/users/6f789b1d3332413696fabc10a8531881/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.265676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.266001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597474f-042a-4e42-82c9-c454b7c7f268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d1ad0a68e44a67b132601065b89dd6, user_id=6f789b1d3332413696fabc10a8531881, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.266819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.267223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.267584 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] PATH_INFO: `/v3/users/9d4186e846f7462dbbb27bbf1f84e33f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.268743 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597474f-042a-4e42-82c9-c454b7c7f268 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.268920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d4186e846f7462dbbb27bbf1f84e33f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.270958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.277710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:08.279869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1344/2710] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:08 2022] PUT /identity/v3/projects/27d1ad0a68e44a67b132601065b89dd6/users/6f789b1d3332413696fabc10a8531881/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:08.292881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.293224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.293658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.322445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d4186e846f7462dbbb27bbf1f84e33f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:08.326518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] MFA Rules not processed for user `6f789b1d3332413696fabc10a8531881`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:08.327874 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:08.335264 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d4186e846f7462dbbb27bbf1f84e33f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:08.335909 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d4186e846f7462dbbb27bbf1f84e33f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:08.348490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d4186e846f7462dbbb27bbf1f84e33f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:08.355562 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-65cf84da-20ea-4d96-b7b3-77a3ace0e6e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d4186e846f7462dbbb27bbf1f84e33f'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:08.362448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1367/2711] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:08 2022] DELETE /identity/v3/users/9d4186e846f7462dbbb27bbf1f84e33f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:08.369333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.369687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.369948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.390606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:08.391871 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:08.403865 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1345/2712] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:08.416343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.417873 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:08.464548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1368/2713] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:08.475500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.476911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:08.516778 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863fb14-d82a-4ad5-8485-25b7772e4039 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.551121 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e44c-06ff-4347-aea2-188675e850e9 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.555148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.556043 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.556423 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.556786 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.567621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.569044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.574168 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:08.581069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14edd14-9d5c-4363-870e-59d84a7b465d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.581278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1369/2714] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:08.582344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14edd14-9d5c-4363-870e-59d84a7b465d service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.582901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14edd14-9d5c-4363-870e-59d84a7b465d service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.583334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14edd14-9d5c-4363-870e-59d84a7b465d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.656650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14edd14-9d5c-4363-870e-59d84a7b465d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.658144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14edd14-9d5c-4363-870e-59d84a7b465d service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:08.668902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1346/2715] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:08.957868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:08.959619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:08.985737 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fb89b-b837-4f77-baf3-989af658cc9a service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:08.992522 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:08.993369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:08.993748 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:08.994140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] PATH_INFO: `/v3/projects/e1c15ff898954850a76318cc9209a9b4` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:08.995417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c15ff898954850a76318cc9209a9b4)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:08.998263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.041376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:09.076738 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c15ff898954850a76318cc9209a9b4', 'request_id': 'req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.084062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c15ff898954850a76318cc9209a9b4', 'request_id': 'req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.091491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c15ff898954850a76318cc9209a9b4', 'request_id': 'req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.113532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] The token cache is being invalidate because project e1c15ff898954850a76318cc9209a9b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:09.114053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.114837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.119187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1370/2716] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:08 2022] DELETE /identity/v3/projects/e1c15ff898954850a76318cc9209a9b4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:09.122154 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54d13-d689-4ff3-9e78-7727ac5da04e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.122680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54d13-d689-4ff3-9e78-7727ac5da04e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.122680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54d13-d689-4ff3-9e78-7727ac5da04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.147853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d7a54d13-d689-4ff3-9e78-7727ac5da04e None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:09.149153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d7a54d13-d689-4ff3-9e78-7727ac5da04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:09.174602 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.177390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:09.219948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1347/2717] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:09.229195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e022f-36bc-409e-a149-e7e9d31918a2 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.229516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e022f-36bc-409e-a149-e7e9d31918a2 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.229758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e022f-36bc-409e-a149-e7e9d31918a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.251701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59d4bd-de5c-486d-8ea3-97ff8671f8b3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.258002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.258850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.259298 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.259632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] PATH_INFO: `/v3/users/e1d2030cb2f340f2b3f839befd5f8d16` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.260951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d2030cb2f340f2b3f839befd5f8d16)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.262966 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.327166 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d2030cb2f340f2b3f839befd5f8d16'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.341869 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d2030cb2f340f2b3f839befd5f8d16'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.342709 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d2030cb2f340f2b3f839befd5f8d16'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.356368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d2030cb2f340f2b3f839befd5f8d16'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.363622 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6710a17c-ae76-4fd6-9603-81c65fbdb6e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d2030cb2f340f2b3f839befd5f8d16'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:09.370475 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1371/2718] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:09 2022] DELETE /identity/v3/users/e1d2030cb2f340f2b3f839befd5f8d16 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:09.374850 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.375170 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.375464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.387321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-389e022f-36bc-409e-a149-e7e9d31918a2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:09.388625 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-389e022f-36bc-409e-a149-e7e9d31918a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:09.404474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:09.405988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:09.434255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:09.467526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1348/2719] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:09.472692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e022f-36bc-409e-a149-e7e9d31918a2 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.482145 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e022f-36bc-409e-a149-e7e9d31918a2 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.485366 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1372/2720] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:09.488473 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.489489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.490025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.490395 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.491184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:09.491586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.493548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.495148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.496646 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:09.512028 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1349/2721] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:09.516005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.528499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865ecf3-8486-422f-b72d-352195a115e1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.534280 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.535195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.535674 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.536201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.536904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.538869 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.570656 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa63c7a-a3ab-4a8f-b852-032091613bb1 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.575116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.575272 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1350/2722] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:09 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:09.575389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.575613 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.575888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.579367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.586859 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.587635 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:09.589569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.605353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1373/2723] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:09.695259 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b35548-fd66-498a-bb57-a1d4179736e9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.703722 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.704524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.705010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.705213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.705818 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.708502 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.731129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1351/2724] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:09 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:09.738958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.749401 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d547c-5bec-4300-ae0d-4c1fe2901b25 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.755729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.756340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.756618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.756888 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.757537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.759175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.774592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1374/2725] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:09.775492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.788169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49934b49-0131-456d-82e1-bfefe6de3a8c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.793708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f0e19-63f2-4df7-89f8-094b864d856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.794499 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0e19-63f2-4df7-89f8-094b864d856d admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.794973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0e19-63f2-4df7-89f8-094b864d856d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.795121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f0e19-63f2-4df7-89f8-094b864d856d admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.795834 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f0e19-63f2-4df7-89f8-094b864d856d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.797427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f0e19-63f2-4df7-89f8-094b864d856d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.815127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1352/2726] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:16:09.821028 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:09.832832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907143b7-d3b1-415b-b2b8-8049e81b1d88 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:09.837734 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:09.838426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.838684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.838956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] PATH_INFO: `/v3/projects/3095e64d6bdf4f8aac1bb14397cb1b9c/users/2fcc24788b51470f8ecbb9f5a0f28bca/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.861042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3095e64d6bdf4f8aac1bb14397cb1b9c, user_id=2fcc24788b51470f8ecbb9f5a0f28bca, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:09.864521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:09.885345 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1375/2727] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:09 2022] PUT /identity/v3/projects/3095e64d6bdf4f8aac1bb14397cb1b9c/users/2fcc24788b51470f8ecbb9f5a0f28bca/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:09.892776 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:09.893321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:09.893806 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:09.950992 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] MFA Rules not processed for user `2fcc24788b51470f8ecbb9f5a0f28bca`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:09.952421 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:10.023950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1353/2728] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:10.029258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:10.032402 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:10.043343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:10.045706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:10.114417 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1caa02-9d25-4288-95a3-c8b5480ff098 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:10.118701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:10.119370 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:10.119652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:10.119930 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] PATH_INFO: `/v3/projects/ee503e1827984f328f02cf1a583ca4a2` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:10.121075 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee503e1827984f328f02cf1a583ca4a2)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:10.123006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:10.126823 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60475c3a-32b2-4e46-8ab9-2d58c54b005e None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:10.131267 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:10.131876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:10.132164 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:10.132435 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:10.143958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:10.146888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:10.156986 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1354/2729] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:10.169755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:10.178293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee503e1827984f328f02cf1a583ca4a2', 'request_id': 'req-978c8599-4e1f-4112-929b-8e2fab08a318'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.185296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee503e1827984f328f02cf1a583ca4a2', 'request_id': 'req-978c8599-4e1f-4112-929b-8e2fab08a318'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.192374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee503e1827984f328f02cf1a583ca4a2', 'request_id': 'req-978c8599-4e1f-4112-929b-8e2fab08a318'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.207150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] The token cache is being invalidate because project ee503e1827984f328f02cf1a583ca4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:10.207624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.208283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.209803 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1376/2730] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:10 2022] DELETE /identity/v3/projects/ee503e1827984f328f02cf1a583ca4a2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:10.213960 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:10.226047 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c9efe-72b5-4be0-ab12-618448505815 service glance] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:10.238837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:10.295561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:10.296204 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:10.296483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:10.296805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] PATH_INFO: `/v3/users/6f789b1d3332413696fabc10a8531881` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:10.297951 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f789b1d3332413696fabc10a8531881)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:10.299969 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:10.351968 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f789b1d3332413696fabc10a8531881'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.364724 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f789b1d3332413696fabc10a8531881'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.365584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f789b1d3332413696fabc10a8531881'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.366343 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f789b1d3332413696fabc10a8531881'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.375039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1b235cc8-96b8-4e86-a859-ed27ac314204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f789b1d3332413696fabc10a8531881'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.388922 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1355/2731] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:10 2022] DELETE /identity/v3/users/6f789b1d3332413696fabc10a8531881 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:10.720007 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:10.721570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:10.736445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c8599-4e1f-4112-929b-8e2fab08a318 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:10.797655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:10.798281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:10.798559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:10.799301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] PATH_INFO: `/v3/projects/27d1ad0a68e44a67b132601065b89dd6` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:10.800587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d1ad0a68e44a67b132601065b89dd6)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:10.802895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:10.851901 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d1ad0a68e44a67b132601065b89dd6', 'request_id': 'req-30df5d59-127c-444f-8fc9-f3d05bcd398e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.862736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d1ad0a68e44a67b132601065b89dd6', 'request_id': 'req-30df5d59-127c-444f-8fc9-f3d05bcd398e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.869731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d1ad0a68e44a67b132601065b89dd6', 'request_id': 'req-30df5d59-127c-444f-8fc9-f3d05bcd398e'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.890081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] The token cache is being invalidate because project 27d1ad0a68e44a67b132601065b89dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:10.890377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.891059 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-30df5d59-127c-444f-8fc9-f3d05bcd398e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:10.892624 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1377/2732] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:10 2022] DELETE /identity/v3/projects/27d1ad0a68e44a67b132601065b89dd6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:10.898192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc3592-cf25-4c5a-9829-6591ea6f91cc None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:10.898555 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc3592-cf25-4c5a-9829-6591ea6f91cc None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:10.898856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc3592-cf25-4c5a-9829-6591ea6f91cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:10.921853 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-54fc3592-cf25-4c5a-9829-6591ea6f91cc None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:10.923120 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-54fc3592-cf25-4c5a-9829-6591ea6f91cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:11.026445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1356/2733] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:11.033747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c6796-6ddf-485f-8dcc-fde0eef98ced None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.034351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c6796-6ddf-485f-8dcc-fde0eef98ced None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.034351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c6796-6ddf-485f-8dcc-fde0eef98ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.057538 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-ea8c6796-6ddf-485f-8dcc-fde0eef98ced None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:11.058787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ea8c6796-6ddf-485f-8dcc-fde0eef98ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:11.073233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1378/2734] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:11.074433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc3592-cf25-4c5a-9829-6591ea6f91cc None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.086641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc3592-cf25-4c5a-9829-6591ea6f91cc None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.095519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.096160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.096597 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.096948 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.098629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:11.099034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.101163 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.125972 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1357/2735] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:16:11.132211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c6796-6ddf-485f-8dcc-fde0eef98ced None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.149974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c6796-6ddf-485f-8dcc-fde0eef98ced None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.156490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.157147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.157431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.158785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.158785 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.161906 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.212154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1379/2736] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:11 2022] POST /identity/v3/projects => generated 367 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:11.216020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.226796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91e9a9-7437-467e-ae8d-fa798f8bdd47 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.245747 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:11.288751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.289374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.289718 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.289919 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.290899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.293372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.328680 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1358/2737] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:11 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:11.330825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.344089 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ff6f0-f6ff-413a-b6df-058c87b3b9fd admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.349610 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.350241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.350508 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.350782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.351464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.353120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.366895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1380/2738] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:16:11.368361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.378664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5472-774c-40cc-a65c-0f627576788c admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.383562 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6d9bc-d23e-420c-80a9-ecc17c2105a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.384202 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6d9bc-d23e-420c-80a9-ecc17c2105a3 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.384501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6d9bc-d23e-420c-80a9-ecc17c2105a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.384794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6d9bc-d23e-420c-80a9-ecc17c2105a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.385450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6d9bc-d23e-420c-80a9-ecc17c2105a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.387098 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6d9bc-d23e-420c-80a9-ecc17c2105a3 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.399394 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1359/2739] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:11.404548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.416053 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a22f39-245f-4411-ba7c-94f4c4141a77 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.420805 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.421494 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.421773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.422051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] PATH_INFO: `/v3/projects/1aab74668d8d44f68531b4d31cd95adb/users/53d29208b2d645228085b39cbe980562/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.441582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aab74668d8d44f68531b4d31cd95adb, user_id=53d29208b2d645228085b39cbe980562, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.447548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.453689 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:11.467891 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1381/2740] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:11 2022] PUT /identity/v3/projects/1aab74668d8d44f68531b4d31cd95adb/users/53d29208b2d645228085b39cbe980562/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:11.480034 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae06c85-9418-40a2-a71d-a82f668d09a0 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.480316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae06c85-9418-40a2-a71d-a82f668d09a0 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.480591 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae06c85-9418-40a2-a71d-a82f668d09a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.511847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-fae06c85-9418-40a2-a71d-a82f668d09a0 None None] MFA Rules not processed for user `53d29208b2d645228085b39cbe980562`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:11.513092 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-fae06c85-9418-40a2-a71d-a82f668d09a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:11.579595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1360/2741] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:11.585693 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.654284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dace1a-c016-4f02-85c5-0cfff63bf8f1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.659661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.660429 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.660567 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.660810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.661468 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.663498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.693563 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1382/2742] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:11 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:11.698070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae06c85-9418-40a2-a71d-a82f668d09a0 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.711161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae06c85-9418-40a2-a71d-a82f668d09a0 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.775746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.776357 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.776636 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.776907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.777567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.779709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.802846 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1361/2743] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:11 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:11.807484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.819107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d82a-f694-46ac-9eac-2b0ee7d9021a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.824077 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.824710 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.824989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.825263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.825908 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.827705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.845262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1383/2744] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:11.849491 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.860159 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0a0e1-a386-4a40-abef-a467a3ea2c6a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.865274 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.865927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.866208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.866483 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] PATH_INFO: `/v3/projects/cb836841be644e659615455add0e8cea/users/441cbca828f849d28d08dc348fa9fd48/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.883518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb836841be644e659615455add0e8cea, user_id=441cbca828f849d28d08dc348fa9fd48, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.886076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.889526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:11.902050 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1362/2745] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:11 2022] PUT /identity/v3/projects/cb836841be644e659615455add0e8cea/users/441cbca828f849d28d08dc348fa9fd48/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:11.906118 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:11.916340 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562cb02-307f-4a85-90e8-44c52fe52e4c admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:11.975389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3d398-7904-4d78-bd1d-69bbd94be6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:11.975982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3d398-7904-4d78-bd1d-69bbd94be6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:11.976257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3d398-7904-4d78-bd1d-69bbd94be6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:11.976524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3d398-7904-4d78-bd1d-69bbd94be6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:11.977146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3d398-7904-4d78-bd1d-69bbd94be6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:11.978722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3d398-7904-4d78-bd1d-69bbd94be6b0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:11.988955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1384/2746] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:11.993209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:12.005311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29c1b6-6e6c-4dc9-a52c-d3c4b99e71ec admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:12.011690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:12.012351 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:12.012632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:12.012905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] PATH_INFO: `/v3/projects/cb836841be644e659615455add0e8cea/users/441cbca828f849d28d08dc348fa9fd48/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:12.014864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb836841be644e659615455add0e8cea, user_id=441cbca828f849d28d08dc348fa9fd48, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:12.017663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:12.029513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1363/2747] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:11 2022] PUT /identity/v3/projects/cb836841be644e659615455add0e8cea/users/441cbca828f849d28d08dc348fa9fd48/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:12.041668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f93ebe-1389-43a9-bfbe-07177f92b785 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:12.041668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f93ebe-1389-43a9-bfbe-07177f92b785 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:12.041668 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f93ebe-1389-43a9-bfbe-07177f92b785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:12.073524 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-08f93ebe-1389-43a9-bfbe-07177f92b785 None None] MFA Rules not processed for user `441cbca828f849d28d08dc348fa9fd48`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:12.074744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-08f93ebe-1389-43a9-bfbe-07177f92b785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:12.126731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:12.146701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1385/2748] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:12.157803 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:12.159682 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:12.227135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c3d0-adbb-4bcb-96df-854af3f96492 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:12.230797 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:12.231465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:12.232124 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:12.232426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:12.245086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:12.246640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:12.257132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1364/2749] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:13.751821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f93ebe-1389-43a9-bfbe-07177f92b785 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:13.837676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:13.840016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:13.841683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f93ebe-1389-43a9-bfbe-07177f92b785 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:13.846233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:13.846873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:13.847335 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:13.847775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] PATH_INFO: `/v3/users/53d29208b2d645228085b39cbe980562` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:13.855935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d29208b2d645228085b39cbe980562)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:13.857968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:13.907025 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acbdc-a7d7-44fb-beea-e54a2faecb3d service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:13.909816 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d29208b2d645228085b39cbe980562'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:13.910810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:13.911456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:13.911737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:13.912012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:13.913910 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:13.920042 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d29208b2d645228085b39cbe980562'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:13.920809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d29208b2d645228085b39cbe980562'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:13.935587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d29208b2d645228085b39cbe980562'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:13.943238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1e1637f7-de97-41d4-a946-65d43fa6efe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d29208b2d645228085b39cbe980562'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:13.949305 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1386/2750] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:13 2022] DELETE /identity/v3/users/53d29208b2d645228085b39cbe980562 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:13.954133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:13.954601 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:13.955004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:13.978505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:13.979879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:13.984744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:13.986188 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:13.997405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1365/2751] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:14.041373 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1387/2752] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:14.053382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:14.054904 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:14.067105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc606f-dbce-431d-ba72-9ca62e3c44c1 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:14.111213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:14.152464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:14.153184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:14.153534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:14.153880 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:14.164996 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:14.166642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:14.180621 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1366/2753] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:14.477760 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:14.479835 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:14.511348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff414-c595-4de1-b087-4a48bf9d1136 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:14.528410 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:14.529289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:14.529589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:14.529873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] PATH_INFO: `/v3/projects/1aab74668d8d44f68531b4d31cd95adb` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:14.531255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aab74668d8d44f68531b4d31cd95adb)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:14.533342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:14.546279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:14.601395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aab74668d8d44f68531b4d31cd95adb', 'request_id': 'req-6cbef7e4-dba2-4f69-b435-d83c50f975f6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.610035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aab74668d8d44f68531b4d31cd95adb', 'request_id': 'req-6cbef7e4-dba2-4f69-b435-d83c50f975f6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.615188 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aab74668d8d44f68531b4d31cd95adb', 'request_id': 'req-6cbef7e4-dba2-4f69-b435-d83c50f975f6'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.629441 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] The token cache is being invalidate because project 1aab74668d8d44f68531b4d31cd95adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:14.629842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.630750 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.632318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1388/2754] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:14 2022] DELETE /identity/v3/projects/1aab74668d8d44f68531b4d31cd95adb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:14.636314 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:14.647784 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e19b-4de4-43f3-99f0-a09b2f108345 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:14.714764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:14.715450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:14.715796 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:14.716143 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] PATH_INFO: `/v3/users/441cbca828f849d28d08dc348fa9fd48` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:14.717603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=441cbca828f849d28d08dc348fa9fd48)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:14.719973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:14.779209 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '441cbca828f849d28d08dc348fa9fd48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.794603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '441cbca828f849d28d08dc348fa9fd48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.795451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '441cbca828f849d28d08dc348fa9fd48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.796122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '441cbca828f849d28d08dc348fa9fd48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.808650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-677ae86e-6a5d-4555-8a57-6d63af9a9b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '441cbca828f849d28d08dc348fa9fd48'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:14.825608 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1367/2755] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:14 2022] DELETE /identity/v3/users/441cbca828f849d28d08dc348fa9fd48 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:15.160369 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:15.162559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:15.187257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbef7e4-dba2-4f69-b435-d83c50f975f6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:15.262511 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:15.263120 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.263739 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.264066 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] PATH_INFO: `/v3/projects/cb836841be644e659615455add0e8cea` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.265281 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb836841be644e659615455add0e8cea)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:15.267682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:15.321023 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb836841be644e659615455add0e8cea', 'request_id': 'req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:15.327395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb836841be644e659615455add0e8cea', 'request_id': 'req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:15.334914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb836841be644e659615455add0e8cea', 'request_id': 'req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:15.349947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] The token cache is being invalidate because project cb836841be644e659615455add0e8cea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:15.350307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:15.350876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-f4594d48-5bf0-4794-98bd-fdf4b0beeafa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:15.352598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1389/2756] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:15 2022] DELETE /identity/v3/projects/cb836841be644e659615455add0e8cea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:15.357300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b7b91-d8f0-40d6-b414-151edd3f0b3a None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.357800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b7b91-d8f0-40d6-b414-151edd3f0b3a None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.357927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b7b91-d8f0-40d6-b414-151edd3f0b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.380226 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f17b7b91-d8f0-40d6-b414-151edd3f0b3a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:15.381457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f17b7b91-d8f0-40d6-b414-151edd3f0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:15.442820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:15.454405 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1368/2757] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:15.462982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289168-f568-4800-9935-0cd15e0e2b82 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.463311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289168-f568-4800-9935-0cd15e0e2b82 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.463588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289168-f568-4800-9935-0cd15e0e2b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.484866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-1e289168-f568-4800-9935-0cd15e0e2b82 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:15.486568 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1e289168-f568-4800-9935-0cd15e0e2b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:15.499337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1390/2758] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:15.501474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b7b91-d8f0-40d6-b414-151edd3f0b3a None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:15.512911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b7b91-d8f0-40d6-b414-151edd3f0b3a None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:15.517603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:15.518214 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.518497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.518769 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.520224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:15.520567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:15.522166 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:15.535933 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1369/2759] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:15.540990 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e289168-f568-4800-9935-0cd15e0e2b82 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:15.553187 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e289168-f568-4800-9935-0cd15e0e2b82 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:15.559029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:15.559680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.559964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.560532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.561393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:15.563942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:15.582368 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:15.598577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1391/2760] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:15 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:15.602918 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:15.613800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda3f7f-1891-40a6-8d61-46fb9c4cb82b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:15.682332 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:15.682980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.683311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.683552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.684224 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:15.686225 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:15.710575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1370/2761] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:15 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:15.714731 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:15.727826 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88072d94-f3a8-441a-8e41-a608ffe2f99d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:15.732533 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:15.733138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.733415 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.733682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.734320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:15.736062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:15.748124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1392/2762] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:15.752055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:15.762760 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ffc0d-1e55-499a-9aec-9d24de1ded65 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:15.767567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37baaae8-9df2-436a-8371-78eb7eca1a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:15.768195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baaae8-9df2-436a-8371-78eb7eca1a8c admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.768485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baaae8-9df2-436a-8371-78eb7eca1a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.768852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baaae8-9df2-436a-8371-78eb7eca1a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.770313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37baaae8-9df2-436a-8371-78eb7eca1a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:15.771949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37baaae8-9df2-436a-8371-78eb7eca1a8c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:15.786859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1371/2763] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:15.788126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:15.809257 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6c17-389b-4566-b353-a550e8b361b3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:15.814493 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:15.815318 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.815604 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.815942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] PATH_INFO: `/v3/projects/1f2b3cd0d9584e1599285725dd852cc0/users/02b21bf1f53648a5b516eda54aaea5c2/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.832798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2b3cd0d9584e1599285725dd852cc0, user_id=02b21bf1f53648a5b516eda54aaea5c2, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:15.835808 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:15.850195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1393/2764] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:15 2022] PUT /identity/v3/projects/1f2b3cd0d9584e1599285725dd852cc0/users/02b21bf1f53648a5b516eda54aaea5c2/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:15.861676 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5024076-ce6e-4deb-82f5-837f418f259f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:15.862039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5024076-ce6e-4deb-82f5-837f418f259f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:15.862400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5024076-ce6e-4deb-82f5-837f418f259f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:15.892610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c5024076-ce6e-4deb-82f5-837f418f259f None None] MFA Rules not processed for user `02b21bf1f53648a5b516eda54aaea5c2`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:15.894712 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c5024076-ce6e-4deb-82f5-837f418f259f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:15.977882 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1372/2765] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:15.982277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.069406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611083a-4263-41a4-8c28-518737a72266 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.075035 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.075857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.076449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.076449 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.077245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:16.079357 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:16.109967 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1394/2766] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:15 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:16.114201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5024076-ce6e-4deb-82f5-837f418f259f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.125411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5024076-ce6e-4deb-82f5-837f418f259f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.200517 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:16.225666 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.226300 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.226578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.226878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.227581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:16.229596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:16.253086 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1373/2767] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:16 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:16.257752 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.281004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71708f49-cc64-4527-92ad-d2dc4726cd17 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.299323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.299323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.299323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.299323 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.299975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:16.303122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:16.317392 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1395/2768] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:16.321779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.333429 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb006d-5b2f-4dbe-b2a4-27f585717feb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.339222 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.340012 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.340308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.340658 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] PATH_INFO: `/v3/projects/eb612348603541f5b4698ce61d29901f/users/838b847bc0e64788a758a058c8020808/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.362147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb612348603541f5b4698ce61d29901f, user_id=838b847bc0e64788a758a058c8020808, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:16.366369 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:16.383157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1374/2769] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:16 2022] PUT /identity/v3/projects/eb612348603541f5b4698ce61d29901f/users/838b847bc0e64788a758a058c8020808/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:16.386590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.398856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce89369-6e61-4e9f-8fd5-ada9cc69b1a8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.476033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60687f68-3a8b-4a06-9819-c956a60212e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.476787 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687f68-3a8b-4a06-9819-c956a60212e9 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.477135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687f68-3a8b-4a06-9819-c956a60212e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.477416 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687f68-3a8b-4a06-9819-c956a60212e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.478144 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687f68-3a8b-4a06-9819-c956a60212e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:16.479810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687f68-3a8b-4a06-9819-c956a60212e9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:16.491103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1396/2770] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:16.494973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.512205 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1dfd82-79f1-4d9d-8a36-f1c36426c79a admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.518649 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.520269 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.521552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.521552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] PATH_INFO: `/v3/projects/eb612348603541f5b4698ce61d29901f/users/838b847bc0e64788a758a058c8020808/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.523785 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb612348603541f5b4698ce61d29901f, user_id=838b847bc0e64788a758a058c8020808, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:16.526063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:16.550907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1375/2771] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:16 2022] PUT /identity/v3/projects/eb612348603541f5b4698ce61d29901f/users/838b847bc0e64788a758a058c8020808/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:16.560575 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfdd9f-9e37-425c-bc12-60225ae01cf4 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.560862 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfdd9f-9e37-425c-bc12-60225ae01cf4 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.561143 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfdd9f-9e37-425c-bc12-60225ae01cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.595695 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-4ebfdd9f-9e37-425c-bc12-60225ae01cf4 None None] MFA Rules not processed for user `838b847bc0e64788a758a058c8020808`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:16.597083 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4ebfdd9f-9e37-425c-bc12-60225ae01cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:16.630338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:16.699866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1397/2772] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:16.711744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.713886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:16.793005 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e571c4c-6b23-47a9-970a-cefa7ee86418 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.797196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.797485 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.797768 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.798044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:16.810079 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:16.811875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:16.827703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1376/2773] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:16.951310 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfdd9f-9e37-425c-bc12-60225ae01cf4 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:16.963595 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfdd9f-9e37-425c-bc12-60225ae01cf4 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:16.967614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763899d-9ccf-4dd3-a7e9-44d31ea0170b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:16.968374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763899d-9ccf-4dd3-a7e9-44d31ea0170b service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:16.968772 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763899d-9ccf-4dd3-a7e9-44d31ea0170b service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:16.969135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763899d-9ccf-4dd3-a7e9-44d31ea0170b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:17.054655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763899d-9ccf-4dd3-a7e9-44d31ea0170b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:17.056446 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763899d-9ccf-4dd3-a7e9-44d31ea0170b service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:17.071379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1398/2774] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:19.760958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:19.829967 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772bedd-a991-45c8-b1e4-a9bb842f7d18 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:19.835565 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:19.836219 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:19.836508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:19.836780 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] PATH_INFO: `/v3/users/02b21bf1f53648a5b516eda54aaea5c2` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:19.837913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b21bf1f53648a5b516eda54aaea5c2)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:19.839908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:19.891667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b21bf1f53648a5b516eda54aaea5c2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:19.893577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:19.907003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b21bf1f53648a5b516eda54aaea5c2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:19.907891 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b21bf1f53648a5b516eda54aaea5c2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:19.908522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b21bf1f53648a5b516eda54aaea5c2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:19.916655 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b21bf1f53648a5b516eda54aaea5c2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:19.934342 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1377/2775] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:19 2022] DELETE /identity/v3/users/02b21bf1f53648a5b516eda54aaea5c2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:19.940034 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:19.940528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:19.940640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:19.968723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:19.970496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:20.039337 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1399/2776] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:20.049114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:20.051121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:20.112150 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2901cd-ac68-4d2b-99b7-2bbbba7e37cb admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:20.116463 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:20.117070 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:20.117352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:20.117633 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:20.126678 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:20.128165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:20.140036 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1378/2777] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:20.417129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:20.418795 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:20.432954 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88857b2d-7742-4bfe-a754-7791a871e457 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:20.437605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:20.438240 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:20.438525 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:20.438800 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] PATH_INFO: `/v3/projects/1f2b3cd0d9584e1599285725dd852cc0` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:20.439993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2b3cd0d9584e1599285725dd852cc0)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:20.442972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:20.492937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2b3cd0d9584e1599285725dd852cc0', 'request_id': 'req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.494054 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:20.500628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2b3cd0d9584e1599285725dd852cc0', 'request_id': 'req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.506407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2b3cd0d9584e1599285725dd852cc0', 'request_id': 'req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.518223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] The token cache is being invalidate because project 1f2b3cd0d9584e1599285725dd852cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:20.518461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.519128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.520730 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1400/2778] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:20 2022] DELETE /identity/v3/projects/1f2b3cd0d9584e1599285725dd852cc0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:20.524861 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:20.534511 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21fb7b-e27a-42db-927d-c2669ba08a66 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:20.602794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:20.603471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:20.603755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:20.604029 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] PATH_INFO: `/v3/users/838b847bc0e64788a758a058c8020808` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:20.605165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=838b847bc0e64788a758a058c8020808)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:20.607112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:20.672642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '838b847bc0e64788a758a058c8020808'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.686510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '838b847bc0e64788a758a058c8020808'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.687388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '838b847bc0e64788a758a058c8020808'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.688018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '838b847bc0e64788a758a058c8020808'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.693356 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:20.695373 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-29dabecf-2ce5-4b3a-b8af-bb6c6dbf78d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '838b847bc0e64788a758a058c8020808'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:20.710777 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1379/2779] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:20 2022] DELETE /identity/v3/users/838b847bc0e64788a758a058c8020808 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:21.015107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.015107 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:21.035474 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e419fe-85e6-4a3f-969e-3d6bf4b8dd8d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.107571 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.108148 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.108425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.108700 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] PATH_INFO: `/v3/projects/eb612348603541f5b4698ce61d29901f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.109845 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb612348603541f5b4698ce61d29901f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.111781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.163390 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb612348603541f5b4698ce61d29901f', 'request_id': 'req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:21.169706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb612348603541f5b4698ce61d29901f', 'request_id': 'req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:21.175251 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb612348603541f5b4698ce61d29901f', 'request_id': 'req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:21.189464 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] The token cache is being invalidate because project eb612348603541f5b4698ce61d29901f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:21.189781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:21.190471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-66f8db40-8ef4-48c2-a6cb-f7aa0f9c7076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:21.193243 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1401/2780] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:21 2022] DELETE /identity/v3/projects/eb612348603541f5b4698ce61d29901f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:21.198571 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3a24a-4049-471f-b2ad-50bbda0abbc4 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.199161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3a24a-4049-471f-b2ad-50bbda0abbc4 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.199161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3a24a-4049-471f-b2ad-50bbda0abbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.220101 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-2ce3a24a-4049-471f-b2ad-50bbda0abbc4 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:21.221298 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2ce3a24a-4049-471f-b2ad-50bbda0abbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:21.285139 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1380/2781] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:21.293197 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfebb4f-d859-43bf-b9d7-713851f3ceb6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.293487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfebb4f-d859-43bf-b9d7-713851f3ceb6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.293938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfebb4f-d859-43bf-b9d7-713851f3ceb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.316883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-edfebb4f-d859-43bf-b9d7-713851f3ceb6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:21.318248 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-edfebb4f-d859-43bf-b9d7-713851f3ceb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:21.326125 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1402/2782] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:21.330730 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3a24a-4049-471f-b2ad-50bbda0abbc4 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.341825 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3a24a-4049-471f-b2ad-50bbda0abbc4 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.346551 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.347212 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.347619 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.347771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.348414 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:21.348717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.350467 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.362896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1381/2783] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:21.367350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfebb4f-d859-43bf-b9d7-713851f3ceb6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.377755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfebb4f-d859-43bf-b9d7-713851f3ceb6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.383608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.384367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.384754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.385029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.385667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.387616 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.418874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1403/2784] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:21 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:21.421854 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.432241 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f1e6-c66a-4a72-9f6b-e56b0ec037ef admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.492144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.492758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.493032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.493302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.493941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.495870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.516771 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1382/2785] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:21 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:21.521961 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.532428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2323312-ef50-4d6e-a9c8-e4d0a0e738a5 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.537029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.537617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.537892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.538158 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.538790 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.540500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.553202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1404/2786] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:21.557021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.567779 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7595b68-12e4-488b-94aa-407e73096800 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.572505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeaf47e-9680-4bc6-8cc4-1a8c6611c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.573144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaf47e-9680-4bc6-8cc4-1a8c6611c3bb admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.573445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaf47e-9680-4bc6-8cc4-1a8c6611c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.573723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaf47e-9680-4bc6-8cc4-1a8c6611c3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.574376 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeaf47e-9680-4bc6-8cc4-1a8c6611c3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.575995 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeaf47e-9680-4bc6-8cc4-1a8c6611c3bb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.586847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1383/2787] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:21.593312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.607146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cd4d0-ca99-48bc-b8b0-0104d4ee80e7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.612397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.613736 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.613919 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.614126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] PATH_INFO: `/v3/projects/99e00f4074da40788f828998e869fe5e/users/aa0daebe760e431aa952883d3d673b36/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.631684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e00f4074da40788f828998e869fe5e, user_id=aa0daebe760e431aa952883d3d673b36, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.636873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.664311 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1405/2788] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:21 2022] PUT /identity/v3/projects/99e00f4074da40788f828998e869fe5e/users/aa0daebe760e431aa952883d3d673b36/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:21.670358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9fbf7-431d-4d6a-812a-b52e4ce9e265 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.670642 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9fbf7-431d-4d6a-812a-b52e4ce9e265 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.670952 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9fbf7-431d-4d6a-812a-b52e4ce9e265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.711510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c9d9fbf7-431d-4d6a-812a-b52e4ce9e265 None None] MFA Rules not processed for user `aa0daebe760e431aa952883d3d673b36`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:21.712736 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c9d9fbf7-431d-4d6a-812a-b52e4ce9e265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:21.757380 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:21.779129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1384/2789] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:21.783837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.789756 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:21.858995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54e8f4-8891-425e-b682-a7a74eaa9d8d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.864451 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.864993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.865284 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.865560 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.866200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.868160 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.896517 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1406/2790] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:21 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:21.900746 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9fbf7-431d-4d6a-812a-b52e4ce9e265 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:21.911937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9fbf7-431d-4d6a-812a-b52e4ce9e265 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:21.968286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:21.968899 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:21.969174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:21.969450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:21.970097 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:21.972053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:21.994014 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1385/2791] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:21 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:21.998060 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.008980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1118984-85ad-48f9-a288-19b3563db886 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.013713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.014307 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.014585 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.014851 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.015548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.017162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.029175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1407/2792] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:22.033211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.043573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7dff-69ab-4f34-922b-22417d996da1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.048613 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df944e43-2444-4cdc-b316-3e661cd67fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.049239 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df944e43-2444-4cdc-b316-3e661cd67fec admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.049527 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df944e43-2444-4cdc-b316-3e661cd67fec admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.049805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df944e43-2444-4cdc-b316-3e661cd67fec admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.050510 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df944e43-2444-4cdc-b316-3e661cd67fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.052745 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df944e43-2444-4cdc-b316-3e661cd67fec admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.063296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1386/2793] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:22.068317 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.078445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f2786-697d-4337-9855-719a30d1e946 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.083931 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.084586 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.084870 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.085147 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] PATH_INFO: `/v3/projects/a4311a9035224fcf9a7a1968c97d462d/users/52247e09e26c40429f525a7b9664a19d/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.100135 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4311a9035224fcf9a7a1968c97d462d, user_id=52247e09e26c40429f525a7b9664a19d, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.102817 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.116707 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1408/2794] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:22 2022] PUT /identity/v3/projects/a4311a9035224fcf9a7a1968c97d462d/users/52247e09e26c40429f525a7b9664a19d/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:22.127129 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3164bdf-822d-490b-9d84-1ceb159c3b46 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.127400 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3164bdf-822d-490b-9d84-1ceb159c3b46 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.127765 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3164bdf-822d-490b-9d84-1ceb159c3b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.157563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-d3164bdf-822d-490b-9d84-1ceb159c3b46 None None] MFA Rules not processed for user `52247e09e26c40429f525a7b9664a19d`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:22.158837 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-d3164bdf-822d-490b-9d84-1ceb159c3b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:22.233824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1387/2795] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:22.238650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.304200 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eda3c5-7825-4069-aeb2-3ce16cda48c3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.309044 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.309653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.309937 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.310296 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.310885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.312893 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.340309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1409/2796] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:22 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:22.344573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3164bdf-822d-490b-9d84-1ceb159c3b46 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.355552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3164bdf-822d-490b-9d84-1ceb159c3b46 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.404195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:22.414415 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.415031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.415361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.415637 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.416292 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.418228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.439108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1388/2797] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:22 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:22.443983 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.454282 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecbcbd-c0c6-4523-8920-619826271a74 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.459104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.459722 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.460006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.460283 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.460926 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.462514 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.473490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1410/2798] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:22.477934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.488721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb22a3-9743-4bfd-a135-0cbad4bdb2b1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.493382 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.494109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.494447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.494798 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] PATH_INFO: `/v3/projects/c35ba8e711774cb1905d017018b4f03f/users/f420be205a5b4bd8a21c14fa16595cf7/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.510082 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35ba8e711774cb1905d017018b4f03f, user_id=f420be205a5b4bd8a21c14fa16595cf7, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.512828 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.527792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1389/2799] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:22 2022] PUT /identity/v3/projects/c35ba8e711774cb1905d017018b4f03f/users/f420be205a5b4bd8a21c14fa16595cf7/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:22.532949 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.549564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060120d-df68-4453-b721-8a0fc388fafe admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.576256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:22.608812 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0426ae-954a-4c33-97d0-4b7f915da526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.609426 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0426ae-954a-4c33-97d0-4b7f915da526 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.609711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0426ae-954a-4c33-97d0-4b7f915da526 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.609989 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0426ae-954a-4c33-97d0-4b7f915da526 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.610658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0426ae-954a-4c33-97d0-4b7f915da526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.612362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0426ae-954a-4c33-97d0-4b7f915da526 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.624395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1411/2800] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:16:22.628560 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.639503 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0e6d4-9ba3-4d35-886c-bb75a925f1ab admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.644002 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.644648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.644920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.645191 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] PATH_INFO: `/v3/projects/c35ba8e711774cb1905d017018b4f03f/users/f420be205a5b4bd8a21c14fa16595cf7/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.647099 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35ba8e711774cb1905d017018b4f03f, user_id=f420be205a5b4bd8a21c14fa16595cf7, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.649705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.660687 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1390/2801] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:22 2022] PUT /identity/v3/projects/c35ba8e711774cb1905d017018b4f03f/users/f420be205a5b4bd8a21c14fa16595cf7/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:22.671434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.671723 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.671998 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.704619 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] MFA Rules not processed for user `f420be205a5b4bd8a21c14fa16595cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:22.705965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:22.784353 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1412/2802] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:22.795557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.797322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:22.858495 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7395ac4-b061-46c4-af8d-7dbd2e1a74f9 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.862365 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.863691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.864144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.864424 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:22.873836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:22.875286 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:22.884790 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1391/2803] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:22.931165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:22.932614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:22.944773 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af565d-9c2c-4f3b-94e9-826b2570fc48 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:22.948351 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86db800-322b-400c-9790-c8fc1361ea5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:22.948951 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86db800-322b-400c-9790-c8fc1361ea5e service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:22.949231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86db800-322b-400c-9790-c8fc1361ea5e service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:22.949502 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86db800-322b-400c-9790-c8fc1361ea5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:23.011678 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86db800-322b-400c-9790-c8fc1361ea5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:23.013098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86db800-322b-400c-9790-c8fc1361ea5e service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:23.022635 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1413/2804] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:26.010022 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:26.073729 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7a3f-44d6-46e9-be35-9f58378e0c80 service cinder] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:26.078250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:26.078885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:26.079195 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:26.079474 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] PATH_INFO: `/v3/users/aa0daebe760e431aa952883d3d673b36` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:26.080620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0daebe760e431aa952883d3d673b36)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:26.083461 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:26.135081 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0daebe760e431aa952883d3d673b36'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.137010 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:26.149459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0daebe760e431aa952883d3d673b36'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.150118 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0daebe760e431aa952883d3d673b36'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.150723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0daebe760e431aa952883d3d673b36'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.157980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0daebe760e431aa952883d3d673b36'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.171153 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1392/2805] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:26 2022] DELETE /identity/v3/users/aa0daebe760e431aa952883d3d673b36 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:26.175852 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:26.176331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:26.176827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:26.196182 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:26.197374 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:26.245233 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:26.261669 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1414/2806] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:26.272264 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:26.273695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:26.336169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747edf-55f0-444b-bd9b-fe958f66fd0b admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:26.339902 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:26.340627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:26.340914 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:26.341187 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:26.351161 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:26.352595 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:26.362078 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1393/2807] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:26.643744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:26.645892 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:26.659801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd1098-6bd3-45f6-a7df-af87caa82f8a None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:26.664878 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:26.666971 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:26.667453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:26.667970 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] PATH_INFO: `/v3/projects/99e00f4074da40788f828998e869fe5e` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:26.669362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e00f4074da40788f828998e869fe5e)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:26.671479 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:26.721189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e00f4074da40788f828998e869fe5e', 'request_id': 'req-34307170-007b-42a7-bac7-033e6fe5ed7d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.728269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e00f4074da40788f828998e869fe5e', 'request_id': 'req-34307170-007b-42a7-bac7-033e6fe5ed7d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.735727 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e00f4074da40788f828998e869fe5e', 'request_id': 'req-34307170-007b-42a7-bac7-033e6fe5ed7d'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.747958 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] The token cache is being invalidate because project 99e00f4074da40788f828998e869fe5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:26.748247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.748868 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.750378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1415/2808] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:26 2022] DELETE /identity/v3/projects/99e00f4074da40788f828998e869fe5e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:26.755576 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:26.766867 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62a258-cffa-4f4c-9979-0cc05910022b service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:26.829720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:26.830443 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:26.830741 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:26.831004 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] PATH_INFO: `/v3/users/52247e09e26c40429f525a7b9664a19d` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:26.832486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52247e09e26c40429f525a7b9664a19d)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:26.834888 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:26.886088 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52247e09e26c40429f525a7b9664a19d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.899240 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52247e09e26c40429f525a7b9664a19d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.899927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52247e09e26c40429f525a7b9664a19d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.900525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52247e09e26c40429f525a7b9664a19d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.907199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:26.909548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52247e09e26c40429f525a7b9664a19d'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:26.932586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1394/2809] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:26 2022] DELETE /identity/v3/users/52247e09e26c40429f525a7b9664a19d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:27.228204 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:27.230505 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:27.252798 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34307170-007b-42a7-bac7-033e6fe5ed7d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:27.322269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:27.323186 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:27.323472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:27.323744 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] PATH_INFO: `/v3/projects/a4311a9035224fcf9a7a1968c97d462d` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:27.324899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4311a9035224fcf9a7a1968c97d462d)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:27.326775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:27.378094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4311a9035224fcf9a7a1968c97d462d', 'request_id': 'req-8e550336-205a-45c1-b88c-8d2286580cde'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.384648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4311a9035224fcf9a7a1968c97d462d', 'request_id': 'req-8e550336-205a-45c1-b88c-8d2286580cde'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.391344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4311a9035224fcf9a7a1968c97d462d', 'request_id': 'req-8e550336-205a-45c1-b88c-8d2286580cde'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.405012 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] The token cache is being invalidate because project a4311a9035224fcf9a7a1968c97d462d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:27.405304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.406029 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.407717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1416/2810] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:27 2022] DELETE /identity/v3/projects/a4311a9035224fcf9a7a1968c97d462d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:27.413169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:27.414631 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:27.434027 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a4c32-500d-42c7-8c4f-234eff392425 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:27.508639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:27.509344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:27.509606 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:27.509893 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] PATH_INFO: `/v3/users/f420be205a5b4bd8a21c14fa16595cf7` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:27.511107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f420be205a5b4bd8a21c14fa16595cf7)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:27.513038 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:27.564827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f420be205a5b4bd8a21c14fa16595cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.577149 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f420be205a5b4bd8a21c14fa16595cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.577860 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f420be205a5b4bd8a21c14fa16595cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.578545 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f420be205a5b4bd8a21c14fa16595cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.586705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-081618b9-545f-4029-afc8-365e3e0e497d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f420be205a5b4bd8a21c14fa16595cf7'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:27.603578 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1395/2811] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:27 2022] DELETE /identity/v3/users/f420be205a5b4bd8a21c14fa16595cf7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:27.895304 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:27.896809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:27.905106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:27.912235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e550336-205a-45c1-b88c-8d2286580cde admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:27.972555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:27.973137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:27.973377 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:27.973653 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] PATH_INFO: `/v3/projects/c35ba8e711774cb1905d017018b4f03f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:27.974784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35ba8e711774cb1905d017018b4f03f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:27.976793 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.026920 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35ba8e711774cb1905d017018b4f03f', 'request_id': 'req-49cab537-a968-4d53-bc40-42d8562db77a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:28.033545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35ba8e711774cb1905d017018b4f03f', 'request_id': 'req-49cab537-a968-4d53-bc40-42d8562db77a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:28.042006 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35ba8e711774cb1905d017018b4f03f', 'request_id': 'req-49cab537-a968-4d53-bc40-42d8562db77a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:28.055435 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] The token cache is being invalidate because project c35ba8e711774cb1905d017018b4f03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:28.055724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:28.056336 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-49cab537-a968-4d53-bc40-42d8562db77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:28.057830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1417/2812] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:27 2022] DELETE /identity/v3/projects/c35ba8e711774cb1905d017018b4f03f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:28.063956 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12decec8-9734-4995-8ba1-e17a1de1db9b None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.064316 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12decec8-9734-4995-8ba1-e17a1de1db9b None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.064660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12decec8-9734-4995-8ba1-e17a1de1db9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.085327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-12decec8-9734-4995-8ba1-e17a1de1db9b None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:28.086534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-12decec8-9734-4995-8ba1-e17a1de1db9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:28.160645 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1396/2813] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:28.169094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdf2bc-2e9f-4f7c-8bbe-463a14f52e3d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.169382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdf2bc-2e9f-4f7c-8bbe-463a14f52e3d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.169658 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdf2bc-2e9f-4f7c-8bbe-463a14f52e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.189979 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-fecdf2bc-2e9f-4f7c-8bbe-463a14f52e3d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:28.191716 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-fecdf2bc-2e9f-4f7c-8bbe-463a14f52e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:28.200155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1418/2814] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:28.204690 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12decec8-9734-4995-8ba1-e17a1de1db9b None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.215493 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12decec8-9734-4995-8ba1-e17a1de1db9b None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.220496 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.221095 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.221374 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.221663 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.222297 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:28.222677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.224250 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.226705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:28.241213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1397/2815] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:16:28.242549 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdf2bc-2e9f-4f7c-8bbe-463a14f52e3d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.251815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdf2bc-2e9f-4f7c-8bbe-463a14f52e3d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.258223 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.258818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.259124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.259397 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.260024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.261938 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.289403 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1419/2816] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:28 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:28.294179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.305068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7086b1-8435-476c-b9d5-9ea5bb20b3f5 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.361090 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.361703 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.361984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.362260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.362905 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.365381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.387406 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1398/2817] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:28 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:28.391590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.402178 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693d1ca-56fb-48f0-a324-8590c76aa1a1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.406964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.407593 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.407869 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.408137 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.408770 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.410490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.421485 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1420/2818] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:28.426277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.436940 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2125f9-ed2c-4007-a742-8aecaefaac47 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.442677 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939cf55-91c8-425c-b764-0024981e966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.443381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939cf55-91c8-425c-b764-0024981e966a admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.443653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939cf55-91c8-425c-b764-0024981e966a admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.443920 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939cf55-91c8-425c-b764-0024981e966a admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.444585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939cf55-91c8-425c-b764-0024981e966a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.446183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939cf55-91c8-425c-b764-0024981e966a admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.458152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1399/2819] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:28.462405 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.472883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dd454-90e5-4a8a-b556-67f1973ad17f admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.477679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.478356 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.478651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.478924 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] PATH_INFO: `/v3/projects/52bed981c30f4b1cae06bf8de57cd534/users/5910ba200904416097a0bbd1dcb3474a/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.494058 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bed981c30f4b1cae06bf8de57cd534, user_id=5910ba200904416097a0bbd1dcb3474a, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.496769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.511348 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1421/2820] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:28 2022] PUT /identity/v3/projects/52bed981c30f4b1cae06bf8de57cd534/users/5910ba200904416097a0bbd1dcb3474a/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:28.522862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f6c6-fa94-4295-9c1c-2b14f0b0a572 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.524441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f6c6-fa94-4295-9c1c-2b14f0b0a572 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.524441 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f6c6-fa94-4295-9c1c-2b14f0b0a572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.555361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f166f6c6-fa94-4295-9c1c-2b14f0b0a572 None None] MFA Rules not processed for user `5910ba200904416097a0bbd1dcb3474a`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:28.556607 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f166f6c6-fa94-4295-9c1c-2b14f0b0a572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:28.653326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1400/2821] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:28.658206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.697889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:28.739953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c138e-594b-4c76-8541-2e2856bf290d admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.749285 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.749982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.750331 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.750721 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.751487 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.759667 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.783676 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1422/2822] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:28 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:28.788177 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166f6c6-fa94-4295-9c1c-2b14f0b0a572 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.799456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166f6c6-fa94-4295-9c1c-2b14f0b0a572 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.867055 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.867694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.867971 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.868242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.868870 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.870749 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.881534 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:28.894042 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1401/2823] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:28 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:28.898679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.909467 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bc7d8-944f-4646-9d53-c2c664b7ed9a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.914564 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.915211 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.915486 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.915753 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.916388 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.917974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.928605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1423/2824] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:28.932889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.944393 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572411-4c34-4eb8-b518-34af6493a065 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:28.949475 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:28.950136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:28.950477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:28.950787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] PATH_INFO: `/v3/projects/5b60e4aed4954200a6c96815b15d6e21/users/8e9094231f964d298eb00295cc935562/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:28.968340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b60e4aed4954200a6c96815b15d6e21, user_id=8e9094231f964d298eb00295cc935562, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:28.971127 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:28.984748 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1402/2825] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:28 2022] PUT /identity/v3/projects/5b60e4aed4954200a6c96815b15d6e21/users/8e9094231f964d298eb00295cc935562/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:28.989046 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:28.999947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f5a2-c7f6-4cc3-a186-10c0cbe2cad0 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:29.060581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93326488-0f8d-4cd1-a781-f8713ebc5cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:29.061176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326488-0f8d-4cd1-a781-f8713ebc5cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:29.061453 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326488-0f8d-4cd1-a781-f8713ebc5cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:29.061719 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326488-0f8d-4cd1-a781-f8713ebc5cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:29.062379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93326488-0f8d-4cd1-a781-f8713ebc5cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:29.064163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93326488-0f8d-4cd1-a781-f8713ebc5cd3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:29.075341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1424/2826] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:29.080039 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:29.091794 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edef17c-997c-42e3-90af-55e49e772904 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:29.096431 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:29.097076 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:29.097352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:29.098321 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] PATH_INFO: `/v3/projects/5b60e4aed4954200a6c96815b15d6e21/users/8e9094231f964d298eb00295cc935562/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:29.100577 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b60e4aed4954200a6c96815b15d6e21, user_id=8e9094231f964d298eb00295cc935562, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:29.103211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:29.113593 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1403/2827] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:29 2022] PUT /identity/v3/projects/5b60e4aed4954200a6c96815b15d6e21/users/8e9094231f964d298eb00295cc935562/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:29.181561 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:29.181836 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:29.182100 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:29.211173 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] MFA Rules not processed for user `8e9094231f964d298eb00295cc935562`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:29.212384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:29.282711 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1425/2828] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:29.294132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:29.296412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:29.354949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f45fa3-23f2-4270-bd2a-45a47c24dd2f admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:29.358584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e33383-8a95-4658-9ca3-4e2730dfcfcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:29.359221 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e33383-8a95-4658-9ca3-4e2730dfcfcd service cinder] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:29.359501 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e33383-8a95-4658-9ca3-4e2730dfcfcd service cinder] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:29.359770 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e33383-8a95-4658-9ca3-4e2730dfcfcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:29.415602 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e33383-8a95-4658-9ca3-4e2730dfcfcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:29.417033 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e33383-8a95-4658-9ca3-4e2730dfcfcd service cinder] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:29.426171 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1404/2829] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:32.081577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:32.083280 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:32.111384 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:32.150082 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc24e1-615f-4937-9c6d-b58965773bf1 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:32.154598 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:32.155496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:32.155780 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:32.156055 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] PATH_INFO: `/v3/users/274e57f093d24099831bd536e7b3a02c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:32.157175 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=274e57f093d24099831bd536e7b3a02c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:32.159106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:32.206698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274e57f093d24099831bd536e7b3a02c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.215623 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274e57f093d24099831bd536e7b3a02c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.216380 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274e57f093d24099831bd536e7b3a02c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.230592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274e57f093d24099831bd536e7b3a02c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.237681 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274e57f093d24099831bd536e7b3a02c'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.244162 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1426/2830] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:32 2022] DELETE /identity/v3/users/274e57f093d24099831bd536e7b3a02c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:32.248462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:32.248958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:32.249132 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:32.270541 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:32.271816 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:32.335660 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1405/2831] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:32.345532 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:32.346974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:32.408126 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687834d7-8393-435c-b01d-ec09a88da7a1 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:32.412701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:32.413314 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:32.413606 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:32.413876 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:32.423578 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:32.425201 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:32.435637 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1427/2832] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:32.728464 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:32.730235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:32.749756 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda8e31-cdd1-4726-810b-5b293fdf3d62 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:32.762477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:32.763169 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:32.763450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:32.763721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] PATH_INFO: `/v3/projects/402f1e46cc964d499eb67f83ed314f4b` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:32.764852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=402f1e46cc964d499eb67f83ed314f4b)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:32.766751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:32.774418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:32.820210 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '402f1e46cc964d499eb67f83ed314f4b', 'request_id': 'req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.826379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '402f1e46cc964d499eb67f83ed314f4b', 'request_id': 'req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.834427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '402f1e46cc964d499eb67f83ed314f4b', 'request_id': 'req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.845228 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] The token cache is being invalidate because project 402f1e46cc964d499eb67f83ed314f4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:32.845515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.846152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:32.847692 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1406/2833] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:32 2022] DELETE /identity/v3/projects/402f1e46cc964d499eb67f83ed314f4b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:33.712942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:33.714419 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:33.768149 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:33.774944 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5c22c-b0a1-4eda-8401-f41235bc60df service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:33.830895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c789bb-ba85-4d7a-b11b-5ca033560671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:33.831539 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c789bb-ba85-4d7a-b11b-5ca033560671 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:33.831818 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c789bb-ba85-4d7a-b11b-5ca033560671 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:33.832258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c789bb-ba85-4d7a-b11b-5ca033560671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:33.834350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f3c789bb-ba85-4d7a-b11b-5ca033560671 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:33.900628 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c789bb-ba85-4d7a-b11b-5ca033560671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:33.901982 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c789bb-ba85-4d7a-b11b-5ca033560671 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:33.911853 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1428/2834] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:40.989584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:40.991024 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:41.058506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa812-cfd7-459e-9a6d-d6e0e0c7ae98 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:41.063594 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:41.064232 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:41.064513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:41.064786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] PATH_INFO: `/v3/users/5910ba200904416097a0bbd1dcb3474a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:41.066116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5910ba200904416097a0bbd1dcb3474a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:41.068513 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:41.119620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5910ba200904416097a0bbd1dcb3474a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.133108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5910ba200904416097a0bbd1dcb3474a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.133827 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5910ba200904416097a0bbd1dcb3474a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.134433 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5910ba200904416097a0bbd1dcb3474a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.141632 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5910ba200904416097a0bbd1dcb3474a'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.156071 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1407/2835] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:40 2022] DELETE /identity/v3/users/5910ba200904416097a0bbd1dcb3474a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:41.160614 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021374-4753-4972-8568-73f805e80060 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:41.161292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021374-4753-4972-8568-73f805e80060 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:41.161292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021374-4753-4972-8568-73f805e80060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:41.181108 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2a021374-4753-4972-8568-73f805e80060 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:41.182339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2a021374-4753-4972-8568-73f805e80060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:41.245320 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1429/2836] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:41.255839 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:41.257251 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:41.271859 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:41.318661 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f0ba5-f00f-40bb-9e8c-3ae294531ee1 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:41.322519 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:41.323160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:41.323446 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:41.323721 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:41.333213 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:41.334786 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:41.344742 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1408/2837] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:41.621769 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a021374-4753-4972-8568-73f805e80060 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:41.623391 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2a021374-4753-4972-8568-73f805e80060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:41.640640 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a021374-4753-4972-8568-73f805e80060 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:41.647118 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:41.648133 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:41.648612 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:41.648911 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] PATH_INFO: `/v3/projects/52bed981c30f4b1cae06bf8de57cd534` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:41.650367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52bed981c30f4b1cae06bf8de57cd534)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:41.652582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:41.699842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:41.701062 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52bed981c30f4b1cae06bf8de57cd534', 'request_id': 'req-d682952e-a7ac-4caf-abf9-0d2837c050d5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.708887 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52bed981c30f4b1cae06bf8de57cd534', 'request_id': 'req-d682952e-a7ac-4caf-abf9-0d2837c050d5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.714583 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52bed981c30f4b1cae06bf8de57cd534', 'request_id': 'req-d682952e-a7ac-4caf-abf9-0d2837c050d5'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.728346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] The token cache is being invalidate because project 52bed981c30f4b1cae06bf8de57cd534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:41.728638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.729274 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.730757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1430/2838] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:41 2022] DELETE /identity/v3/projects/52bed981c30f4b1cae06bf8de57cd534 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:41.734708 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:41.746299 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cab44-7eb1-48c6-9318-05dded494a1d service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:41.803720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:41.804388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:41.804664 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:41.804937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] PATH_INFO: `/v3/users/8e9094231f964d298eb00295cc935562` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:41.806067 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e9094231f964d298eb00295cc935562)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:41.808021 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:41.858341 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e9094231f964d298eb00295cc935562'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.870179 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e9094231f964d298eb00295cc935562'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.870783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e9094231f964d298eb00295cc935562'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.871390 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e9094231f964d298eb00295cc935562'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.879252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-1d1fd6b5-d47d-46dd-817d-a517ea1b1eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e9094231f964d298eb00295cc935562'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:41.892387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1409/2839] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:41 2022] DELETE /identity/v3/users/8e9094231f964d298eb00295cc935562 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:42.166457 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.167921 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:42.183830 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682952e-a7ac-4caf-abf9-0d2837c050d5 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.243594 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.244242 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.244521 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.244796 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] PATH_INFO: `/v3/projects/5b60e4aed4954200a6c96815b15d6e21` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.245942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b60e4aed4954200a6c96815b15d6e21)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.247914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:42.299463 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b60e4aed4954200a6c96815b15d6e21', 'request_id': 'req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:42.306941 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b60e4aed4954200a6c96815b15d6e21', 'request_id': 'req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:42.312839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b60e4aed4954200a6c96815b15d6e21', 'request_id': 'req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:42.328321 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] The token cache is being invalidate because project 5b60e4aed4954200a6c96815b15d6e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:42.328643 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:42.329302 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-4456bbc2-fdbd-4bfa-bd9e-47e7a2e7ab9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:42.330801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1431/2840] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:42 2022] DELETE /identity/v3/projects/5b60e4aed4954200a6c96815b15d6e21 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:42.338515 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4bc91-c300-495a-826f-5dd4d9ff720c None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.339031 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4bc91-c300-495a-826f-5dd4d9ff720c None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.339371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4bc91-c300-495a-826f-5dd4d9ff720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.365804 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-74d4bc91-c300-495a-826f-5dd4d9ff720c None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:42.367019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-74d4bc91-c300-495a-826f-5dd4d9ff720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:42.397840 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:42.428623 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1410/2841] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:42.437070 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4a255-d55b-44f7-92bd-60456517ff84 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.437395 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4a255-d55b-44f7-92bd-60456517ff84 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.437735 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4a255-d55b-44f7-92bd-60456517ff84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.459458 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-a6d4a255-d55b-44f7-92bd-60456517ff84 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:42.460705 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a6d4a255-d55b-44f7-92bd-60456517ff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:42.468720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1432/2842] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:42.472950 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4bc91-c300-495a-826f-5dd4d9ff720c None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.485211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4bc91-c300-495a-826f-5dd4d9ff720c None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.491610 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.492242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.492516 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.492787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.493447 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:42.493751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.495371 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:42.509260 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1411/2843] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:42.513914 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4a255-d55b-44f7-92bd-60456517ff84 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.524863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4a255-d55b-44f7-92bd-60456517ff84 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.531122 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.531825 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.532174 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.532484 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.533185 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.535238 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:42.565962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1433/2844] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:42 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:42.571249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.581480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c775d71-4b2c-4564-a534-4cad289bb0ef admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.645230 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.645841 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.646122 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.646387 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.647032 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.648966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:42.670725 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1412/2845] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:42 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:42.674988 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.686409 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb5a2d-118f-454d-af39-b2030d94493e admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.691292 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.692570 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.692889 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.693165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.693810 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.695434 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:42.705588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1434/2846] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:42.709879 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.720358 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2927d9d-e173-40ef-af8c-6dda00bfb948 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.725412 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e498ee-5ca2-470d-a95b-e0c4fe57f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.726019 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e498ee-5ca2-470d-a95b-e0c4fe57f8de admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.726302 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e498ee-5ca2-470d-a95b-e0c4fe57f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.726575 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e498ee-5ca2-470d-a95b-e0c4fe57f8de admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.727278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e498ee-5ca2-470d-a95b-e0c4fe57f8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.728878 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e498ee-5ca2-470d-a95b-e0c4fe57f8de admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:42.740125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1413/2847] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:42.744617 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.755344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb0fbd-4104-4234-9e4a-8d112832586a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.760015 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.760671 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.760948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.761221 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] PATH_INFO: `/v3/projects/6070bbad180c47059d85c639fab9630c/users/841c3d4113374bbaaa1038a819f56fae/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.775815 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6070bbad180c47059d85c639fab9630c, user_id=841c3d4113374bbaaa1038a819f56fae, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.778471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:42.793024 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1435/2848] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:42 2022] PUT /identity/v3/projects/6070bbad180c47059d85c639fab9630c/users/841c3d4113374bbaaa1038a819f56fae/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:42.806427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.806704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.807018 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.837155 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] MFA Rules not processed for user `841c3d4113374bbaaa1038a819f56fae`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:42.838366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:42.905285 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1414/2849] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:42.917657 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:42.919406 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:42.924701 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:42.980431 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f48bf0-62ce-4ae3-9275-7c7bdf3a01b7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:42.983975 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a6fac-e20d-4d12-9aac-5db5ac288444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:42.984577 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a6fac-e20d-4d12-9aac-5db5ac288444 service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:42.984857 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a6fac-e20d-4d12-9aac-5db5ac288444 service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:42.985130 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a6fac-e20d-4d12-9aac-5db5ac288444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:42.994168 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a6fac-e20d-4d12-9aac-5db5ac288444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:42.995633 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a6fac-e20d-4d12-9aac-5db5ac288444 service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:43.005322 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1436/2850] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:44.521521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:44.524586 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:44.564160 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:44.595506 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4dd37-bacf-41c3-a8b9-6aac568d6d85 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:44.600717 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:44.601348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:44.601627 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:44.601900 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] PATH_INFO: `/v3/users/2fcc24788b51470f8ecbb9f5a0f28bca` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:44.603016 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fcc24788b51470f8ecbb9f5a0f28bca)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:44.604980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:44.652337 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fcc24788b51470f8ecbb9f5a0f28bca'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:44.665512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fcc24788b51470f8ecbb9f5a0f28bca'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:44.666208 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fcc24788b51470f8ecbb9f5a0f28bca'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:44.666824 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fcc24788b51470f8ecbb9f5a0f28bca'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:44.673901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fcc24788b51470f8ecbb9f5a0f28bca'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:44.688322 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1415/2851] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:44 2022] DELETE /identity/v3/users/2fcc24788b51470f8ecbb9f5a0f28bca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:44.693150 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:44.693706 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:44.693840 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:44.713139 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:44.714344 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:44.779123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1437/2852] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:44.789465 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:44.790901 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:44.857787 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd90a-d79a-4cc9-806b-df25674bda21 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:44.861538 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e4630-5285-471f-8e4b-9b67984e5497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:44.862136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e4630-5285-471f-8e4b-9b67984e5497 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:44.862411 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e4630-5285-471f-8e4b-9b67984e5497 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:44.862685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e4630-5285-471f-8e4b-9b67984e5497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:44.872539 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e4630-5285-471f-8e4b-9b67984e5497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:44.874136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e4630-5285-471f-8e4b-9b67984e5497 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:44.884289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1416/2853] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:45.157683 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.159177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.171466 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954cadb-b96c-4c97-acb0-c22a074093d2 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.176922 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.177935 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.178119 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.178393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] PATH_INFO: `/v3/projects/3095e64d6bdf4f8aac1bb14397cb1b9c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.179596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3095e64d6bdf4f8aac1bb14397cb1b9c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.181547 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.230991 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3095e64d6bdf4f8aac1bb14397cb1b9c', 'request_id': 'req-bce348a5-6906-42a7-9585-5f2779560aaa'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.237212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3095e64d6bdf4f8aac1bb14397cb1b9c', 'request_id': 'req-bce348a5-6906-42a7-9585-5f2779560aaa'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.243581 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3095e64d6bdf4f8aac1bb14397cb1b9c', 'request_id': 'req-bce348a5-6906-42a7-9585-5f2779560aaa'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.255899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] The token cache is being invalidate because project 3095e64d6bdf4f8aac1bb14397cb1b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:45.256247 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.256875 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-bce348a5-6906-42a7-9585-5f2779560aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.258469 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1438/2854] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:45 2022] DELETE /identity/v3/projects/3095e64d6bdf4f8aac1bb14397cb1b9c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:45.263326 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441152f-e54b-4792-bddf-2eab4c9e0afa None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.263634 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441152f-e54b-4792-bddf-2eab4c9e0afa None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.263906 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441152f-e54b-4792-bddf-2eab4c9e0afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.283293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-7441152f-e54b-4792-bddf-2eab4c9e0afa None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:45.284500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-7441152f-e54b-4792-bddf-2eab4c9e0afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.334528 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:45.349372 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1417/2855] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:45.356987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfafa7f-f281-42ea-adc9-1d1539d8cf7d None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.357199 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfafa7f-f281-42ea-adc9-1d1539d8cf7d None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.357473 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfafa7f-f281-42ea-adc9-1d1539d8cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.377806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-dbfafa7f-f281-42ea-adc9-1d1539d8cf7d None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:45.379033 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dbfafa7f-f281-42ea-adc9-1d1539d8cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.386987 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1439/2856] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:45.391320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441152f-e54b-4792-bddf-2eab4c9e0afa None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.401147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441152f-e54b-4792-bddf-2eab4c9e0afa None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.405720 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.406324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.406611 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.406884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.407569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:45.407875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.409471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.421801 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1418/2857] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:45.426794 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfafa7f-f281-42ea-adc9-1d1539d8cf7d None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.436299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfafa7f-f281-42ea-adc9-1d1539d8cf7d None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.440972 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.441572 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.441856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.442142 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.443004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.444801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.467154 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:45.475608 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1440/2858] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:45 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:45.479442 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.490864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f0d5-f227-49e6-9646-c726a48c45a2 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.547889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.548525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.548809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.549085 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.549737 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.552242 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.559774 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.562131 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.574526 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1419/2859] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:45 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:45.576343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c260-2135-4d26-825c-a0de454d1c9a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.578466 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.581293 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.581942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.582202 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.582483 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] PATH_INFO: `/v3/users/841c3d4113374bbaaa1038a819f56fae` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.583649 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=841c3d4113374bbaaa1038a819f56fae)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.585590 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.588622 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c24b6-c65f-4e62-8feb-cd9ddfa277f4 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.593352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.593949 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.594235 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.594508 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.595182 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.596761 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.608704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1420/2860] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:16:45.612563 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.623934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e0d52-3859-4d89-9d5d-3c776c721edc admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.628696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.630138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.630810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.630810 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.631367 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.633072 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.637138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841c3d4113374bbaaa1038a819f56fae'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.644945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1421/2861] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 19:16:45.646004 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841c3d4113374bbaaa1038a819f56fae'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.646674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841c3d4113374bbaaa1038a819f56fae'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.648698 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.658695 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9370e-3b23-4700-85ff-c96f7180e159 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.659003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841c3d4113374bbaaa1038a819f56fae'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.663521 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8888474-7659-4109-adc3-42d079817e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.664174 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8888474-7659-4109-adc3-42d079817e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.664455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8888474-7659-4109-adc3-42d079817e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.664731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8888474-7659-4109-adc3-42d079817e0c admin admin] PATH_INFO: `/v3/projects/e17392290a004e5f960e7b7502f59373/users/98755f45f02945de929a8eb26a38d124/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.666346 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-38283d3b-cfa6-43d9-934b-450a12d4eec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841c3d4113374bbaaa1038a819f56fae'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:45.673234 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1441/2862] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:45 2022] DELETE /identity/v3/users/841c3d4113374bbaaa1038a819f56fae => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:45.678559 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.678843 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.679140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.681540 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8888474-7659-4109-adc3-42d079817e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17392290a004e5f960e7b7502f59373, user_id=98755f45f02945de929a8eb26a38d124, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.684275 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8888474-7659-4109-adc3-42d079817e0c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.698883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:45.699459 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1422/2863] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:45 2022] PUT /identity/v3/projects/e17392290a004e5f960e7b7502f59373/users/98755f45f02945de929a8eb26a38d124/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:45.700103 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.767848 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1442/2864] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:45.768201 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.768492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.768858 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.777631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.778980 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.799397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] MFA Rules not processed for user `98755f45f02945de929a8eb26a38d124`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:45.800926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.841386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb6686-3033-49ad-bb6f-338692912451 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.845081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.845688 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.845965 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.846237 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.856278 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.857806 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.867974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1443/2865] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:45.872049 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1423/2866] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:45.883587 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:45.885027 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:45.904698 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:45.952600 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1172808-489e-4811-868a-2e444c82cd8c service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:45.957098 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f878d0-5005-4ba2-adc3-d56f96a2589e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:45.958576 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f878d0-5005-4ba2-adc3-d56f96a2589e service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:45.958858 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f878d0-5005-4ba2-adc3-d56f96a2589e service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:45.959165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f878d0-5005-4ba2-adc3-d56f96a2589e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:45.969992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f878d0-5005-4ba2-adc3-d56f96a2589e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:45.971832 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f878d0-5005-4ba2-adc3-d56f96a2589e service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:45.981207 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1444/2867] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:46.006751 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.008329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.032007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b04870-b310-4a5b-b39b-632fd8b97691 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.040522 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.041696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.042304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.042885 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] PATH_INFO: `/v3/users/98755f45f02945de929a8eb26a38d124` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.044980 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98755f45f02945de929a8eb26a38d124)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.048758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.059287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:46.122258 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98755f45f02945de929a8eb26a38d124'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.136445 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98755f45f02945de929a8eb26a38d124'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.137176 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98755f45f02945de929a8eb26a38d124'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.137800 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98755f45f02945de929a8eb26a38d124'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.145650 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98755f45f02945de929a8eb26a38d124'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.160934 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1424/2868] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:16:46 2022] DELETE /identity/v3/users/98755f45f02945de929a8eb26a38d124 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:46.166212 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.166548 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.166824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.185727 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.187254 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.189161 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:46.190360 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.262809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1445/2869] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:46.274174 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.274596 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e4a6c-37a7-48e3-bd4c-eb2c9e91df37 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.275682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.279327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.279961 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.280244 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.280518 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] PATH_INFO: `/v3/projects/6070bbad180c47059d85c639fab9630c` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.281864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6070bbad180c47059d85c639fab9630c)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.284363 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.335430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6070bbad180c47059d85c639fab9630c', 'request_id': 'req-db47567b-5fca-4292-851c-179913127f7c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.337877 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905121-b77d-43ad-802a-33bb5541b730 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.342580 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6070bbad180c47059d85c639fab9630c', 'request_id': 'req-db47567b-5fca-4292-851c-179913127f7c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.351091 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6070bbad180c47059d85c639fab9630c', 'request_id': 'req-db47567b-5fca-4292-851c-179913127f7c'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.362125 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] The token cache is being invalidate because project 6070bbad180c47059d85c639fab9630c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:46.362418 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.363109 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-db47567b-5fca-4292-851c-179913127f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.365413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1425/2870] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:46 2022] DELETE /identity/v3/projects/6070bbad180c47059d85c639fab9630c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:46.370361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640541f-1397-4881-aea3-26c73d4e5ee9 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.370813 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640541f-1397-4881-aea3-26c73d4e5ee9 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.371194 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640541f-1397-4881-aea3-26c73d4e5ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.390537 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.391085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.391376 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.391497 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-2640541f-1397-4881-aea3-26c73d4e5ee9 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:46.391648 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.392694 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2640541f-1397-4881-aea3-26c73d4e5ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.393378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.402641 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:46.425177 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:46.458451 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1426/2871] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:46.466301 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67749c2-c41e-4018-a787-5db808e9e713 None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.466570 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67749c2-c41e-4018-a787-5db808e9e713 None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.466843 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67749c2-c41e-4018-a787-5db808e9e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.467289 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.468819 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.479632 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1446/2872] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:46.486805 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-c67749c2-c41e-4018-a787-5db808e9e713 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:46.488045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-c67749c2-c41e-4018-a787-5db808e9e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.499227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1427/2873] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:46.501140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.512085 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7eb8d-170a-466a-a3f8-ba3b826299f1 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.517072 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.517679 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.517963 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.518231 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] PATH_INFO: `/v3/domains` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.519255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76733) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:46.519255 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.521140 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.534011 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1447/2874] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 19:16:46.538644 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67749c2-c41e-4018-a787-5db808e9e713 None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.548640 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67749c2-c41e-4018-a787-5db808e9e713 None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.554478 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.555133 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.555413 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.555685 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] PATH_INFO: `/v3/projects` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.556323 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.558417 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.587809 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1428/2875] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:46.592394 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.607152 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f19ba-7fc1-431e-a300-f90e848215ca admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.681262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.681913 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.682268 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.682545 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] PATH_INFO: `/v3/users` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.683319 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.685276 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.711929 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1448/2876] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:46.713095 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.724379 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fc8fb-6e47-41a1-b0f8-b4c3a60e4699 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.729403 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.730007 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.730282 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.730552 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.731227 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.732830 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.744328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1429/2877] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:46.748885 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.760873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b3bf1-8955-49d3-98c0-475c7d965beb admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.766784 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.767528 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.767807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.768087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.768903 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.770582 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.773864 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.776245 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.783898 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1449/2878] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:46.790945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e37bd-af4c-4990-859e-43dd61343f45 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.794555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.795945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.796585 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.796892 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.797162 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] PATH_INFO: `/v3/projects/e17392290a004e5f960e7b7502f59373` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.798454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17392290a004e5f960e7b7502f59373)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.800537 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.810471 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0975e-25ac-4dcb-b76e-a23bfc232ee7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:46.816002 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a037cc-dd16-4253-81e6-ceaf52e13228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:46.816719 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a037cc-dd16-4253-81e6-ceaf52e13228 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.817000 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a037cc-dd16-4253-81e6-ceaf52e13228 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.817270 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a037cc-dd16-4253-81e6-ceaf52e13228 admin admin] PATH_INFO: `/v3/projects/af3f2410b6e94037becf20e9c7908820/users/4197287cdb4942f58a59e2016493fd84/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.834957 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a037cc-dd16-4253-81e6-ceaf52e13228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3f2410b6e94037becf20e9c7908820, user_id=4197287cdb4942f58a59e2016493fd84, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:46.837661 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a037cc-dd16-4253-81e6-ceaf52e13228 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:46.851942 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1450/2879] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:46 2022] PUT /identity/v3/projects/af3f2410b6e94037becf20e9c7908820/users/4197287cdb4942f58a59e2016493fd84/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:46.856828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.857235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.857350 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17392290a004e5f960e7b7502f59373', 'request_id': 'req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.857519 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.865453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17392290a004e5f960e7b7502f59373', 'request_id': 'req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.872397 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17392290a004e5f960e7b7502f59373', 'request_id': 'req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.877824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:46.879124 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.883190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] The token cache is being invalidate because project e17392290a004e5f960e7b7502f59373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:16:46.883486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.884147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-95238e48-d1fa-4fa4-afdd-a2b9b93d7947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:16:46.885863 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1430/2880] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:16:46 2022] DELETE /identity/v3/projects/e17392290a004e5f960e7b7502f59373 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:46.890711 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063af9-2448-4309-82a2-ecab87a255db None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.890966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063af9-2448-4309-82a2-ecab87a255db None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.891220 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063af9-2448-4309-82a2-ecab87a255db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:46.910923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-83063af9-2448-4309-82a2-ecab87a255db None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:46.912399 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-83063af9-2448-4309-82a2-ecab87a255db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.943378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1451/2881] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:46.957075 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:46.959839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:46.967875 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:46.971781 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:46.980913 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1431/2882] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:46.987480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51930-f638-41e2-b03c-150460dfb89f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:46.987915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51930-f638-41e2-b03c-150460dfb89f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:46.988197 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51930-f638-41e2-b03c-150460dfb89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.011437 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-2fc51930-f638-41e2-b03c-150460dfb89f None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:47.012668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-2fc51930-f638-41e2-b03c-150460dfb89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:47.021643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1432/2883] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:16:47.025821 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc51930-f638-41e2-b03c-150460dfb89f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:47.026214 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f727d64-7a7f-4e85-b117-2a21894403e6 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.029824 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.030442 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.030720 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.030993 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.034850 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc51930-f638-41e2-b03c-150460dfb89f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.039849 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.040450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.040731 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.040968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.041068 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] PATH_INFO: `/v3/domains` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.041639 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=76732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 19:16:47.041945 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.042407 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.043581 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.052978 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1452/2884] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:16:47.060340 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1433/2885] 10.222.0.94 () {58 vars in 1153 bytes} [Mon Aug 29 19:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 19:16:47.061389 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:47.072597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9e03c-0d7d-4d67-8e8f-6e655a4558b2 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.077303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.077996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.078303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.078652 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.079393 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.081418 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.109956 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1453/2886] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:47 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:47.114966 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:47.125530 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb3f64-d970-4a77-b31a-c8c53163d996 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.190172 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.191020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.191335 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.191628 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.193165 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.195311 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.218599 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1434/2887] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:47 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:47.222490 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:47.233208 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7206a2-664c-4d7a-9f66-33f3d3e739d2 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.237995 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.240230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.240230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.240230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.240827 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.242338 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.257165 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1454/2888] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:47.258384 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:47.268667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e146c-5f4f-45c2-b415-dfc7bbe2843e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.273531 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7fd2f-3634-4c64-b8c8-9765ed4a5b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.274147 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7fd2f-3634-4c64-b8c8-9765ed4a5b89 admin admin] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.274426 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7fd2f-3634-4c64-b8c8-9765ed4a5b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.274700 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7fd2f-3634-4c64-b8c8-9765ed4a5b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.275454 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7fd2f-3634-4c64-b8c8-9765ed4a5b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.277121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7fd2f-3634-4c64-b8c8-9765ed4a5b89 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.288353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1435/2889] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:47.292267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:47.305687 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cc79b-7842-4fda-9c34-72a450c47590 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.310399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.311123 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.311411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.311713 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] PATH_INFO: `/v3/projects/ab3143421f134832813afc985ac1386a/users/223b909a4d624d049c1c813ed391a3d3/roles/b2e24e1a6db641d3a56fde8d3b1d405c` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.327856 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3143421f134832813afc985ac1386a, user_id=223b909a4d624d049c1c813ed391a3d3, role_id=b2e24e1a6db641d3a56fde8d3b1d405c)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.330588 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.343801 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1455/2890] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:47 2022] PUT /identity/v3/projects/ab3143421f134832813afc985ac1386a/users/223b909a4d624d049c1c813ed391a3d3/roles/b2e24e1a6db641d3a56fde8d3b1d405c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:47.356199 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99289fda-3cbb-4d35-9622-b337a7ebc08d None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.356852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99289fda-3cbb-4d35-9622-b337a7ebc08d None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.356852 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99289fda-3cbb-4d35-9622-b337a7ebc08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.389157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-99289fda-3cbb-4d35-9622-b337a7ebc08d None None] MFA Rules not processed for user `223b909a4d624d049c1c813ed391a3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:47.390620 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-99289fda-3cbb-4d35-9622-b337a7ebc08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:47.463889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1436/2891] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:47.475873 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:47.477651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:47.538947 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18993b8-59dc-4156-8f99-fa140b5b7f26 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:47.542853 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee56cf453075431ab2791e15e803d51c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:47.543555 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:47.543839 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:47.544116 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:47.553106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:47.554959 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:47.564253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1456/2892] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:51.280603 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99384c52-0877-4156-b07e-54a1ed1f305f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:51.281157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99384c52-0877-4156-b07e-54a1ed1f305f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:51.281157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99384c52-0877-4156-b07e-54a1ed1f305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:51.314629 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-99384c52-0877-4156-b07e-54a1ed1f305f None None] MFA Rules not processed for user `4197287cdb4942f58a59e2016493fd84`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:51.315886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-99384c52-0877-4156-b07e-54a1ed1f305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:51.356381 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:51.386457 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1437/2893] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:51.390945 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:51.392408 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:51.468597 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:51.476497 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d628b-9c72-4684-b647-ae466c904d9e service cinder] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:51.481425 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:51.482080 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:51.482359 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:51.482631 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:51.483313 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:51.485229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:51.515974 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1457/2894] 10.222.0.94 () {60 vars in 1153 bytes} [Mon Aug 29 19:16:51 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 19:16:51.520758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99384c52-0877-4156-b07e-54a1ed1f305f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:51.532313 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99384c52-0877-4156-b07e-54a1ed1f305f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:51.595557 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:51.596184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:51.596505 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:51.596740 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] PATH_INFO: `/v3/users` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:51.597391 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:51.599452 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:51.622604 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1438/2895] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Aug 29 19:16:51 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 19:16:51.626424 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:51.637828 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf2336-159b-4a4f-8f08-b16c47d147c7 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:51.642339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:51.643132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:51.643253 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:51.643498 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:51.644138 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:51.645866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:51.657488 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1458/2896] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:51.661908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:51.673063 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d1422-634c-408c-bc5b-6c985eb21cff admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:51.679881 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:51.680561 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:51.680938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:51.681137 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] PATH_INFO: `/v3/projects/a4d004040b8d4517a3b1f6b9f6acdf92/users/1a52c04256fa4e99b9d41d3836fa0581/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:51.702231 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d004040b8d4517a3b1f6b9f6acdf92, user_id=1a52c04256fa4e99b9d41d3836fa0581, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:51.705053 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:51.722152 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1439/2897] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:51 2022] PUT /identity/v3/projects/a4d004040b8d4517a3b1f6b9f6acdf92/users/1a52c04256fa4e99b9d41d3836fa0581/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:16:51.726031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:51.743977 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7a72b-b110-4568-9b3e-18a072fc1fa8 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:51.812874 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:51.814094 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:51.814378 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:51.814650 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:51.815362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:51.818031 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:51.829472 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1459/2898] 10.222.0.94 () {58 vars in 1122 bytes} [Mon Aug 29 19:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 19:16:51.834144 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:51.846667 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d847f-fd44-4c9d-8dcf-b5286d5d5109 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:51.851822 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f3d49-9d7d-4bdd-b88e-3efb23fc0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:51.852489 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f3d49-9d7d-4bdd-b88e-3efb23fc0648 admin admin] REQUEST_METHOD: `PUT` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:51.852772 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f3d49-9d7d-4bdd-b88e-3efb23fc0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:51.853057 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f3d49-9d7d-4bdd-b88e-3efb23fc0648 admin admin] PATH_INFO: `/v3/projects/a4d004040b8d4517a3b1f6b9f6acdf92/users/1a52c04256fa4e99b9d41d3836fa0581/roles/89d008d3858c4e1b9ca6c28060b84405` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:51.855029 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f3d49-9d7d-4bdd-b88e-3efb23fc0648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d004040b8d4517a3b1f6b9f6acdf92, user_id=1a52c04256fa4e99b9d41d3836fa0581, role_id=89d008d3858c4e1b9ca6c28060b84405)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:51.858136 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f3d49-9d7d-4bdd-b88e-3efb23fc0648 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:51.869488 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1440/2899] 10.222.0.94 () {60 vars in 1483 bytes} [Mon Aug 29 19:16:51 2022] PUT /identity/v3/projects/a4d004040b8d4517a3b1f6b9f6acdf92/users/1a52c04256fa4e99b9d41d3836fa0581/roles/89d008d3858c4e1b9ca6c28060b84405 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:16:53.617128 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:53.619461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:53.807680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5eb91-d138-4eb6-bb5e-614a9556bad3 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:53.869303 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:53.899755 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:53.900382 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:53.900664 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:53.900933 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:53.908444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:53.924996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:53.927189 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:53.938087 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1460/2900] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:56.827471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55db336-0122-4f81-98f1-57135cb072be None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:56.828096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55db336-0122-4f81-98f1-57135cb072be None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:56.828096 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55db336-0122-4f81-98f1-57135cb072be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:56.868477 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-e55db336-0122-4f81-98f1-57135cb072be None None] MFA Rules not processed for user `1a52c04256fa4e99b9d41d3836fa0581`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:56.870003 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-e55db336-0122-4f81-98f1-57135cb072be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:56.959344 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1441/2901] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:56.967718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:56.969516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.043696 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbf97b-fe9e-403e-a245-1969c63aab0a service placement] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:57.048531 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d9d979b6539460e82759320c3d96a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:57.049198 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.049682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.049925 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.119651 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:57.121104 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:57.131596 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1461/2902] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:16:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:16:57.257978 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f030f7-faaf-4aa6-bbca-825fe21ece7f None None] REQUEST_METHOD: `POST` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.258462 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f030f7-faaf-4aa6-bbca-825fe21ece7f None None] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.258752 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f030f7-faaf-4aa6-bbca-825fe21ece7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.277876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.auth.core [None req-f0f030f7-faaf-4aa6-bbca-825fe21ece7f None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:16:57.279114 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f0f030f7-faaf-4aa6-bbca-825fe21ece7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.289278 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:16:57.338525 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1442/2903] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:16:57.385513 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:16:57.386917 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:16:57.459448 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3f5f-cf05-405e-8e4a-df849ef8a3c0 service glance] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:16:57.464235 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42f35-c8fc-430c-8aa2-fd80682da43b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:16:57.464837 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42f35-c8fc-430c-8aa2-fd80682da43b service swift] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:16:57.465115 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42f35-c8fc-430c-8aa2-fd80682da43b service swift] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:16:57.465386 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42f35-c8fc-430c-8aa2-fd80682da43b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:16:57.473932 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42f35-c8fc-430c-8aa2-fd80682da43b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:16:57.475565 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42f35-c8fc-430c-8aa2-fd80682da43b service swift] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:16:57.483523 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1462/2904] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:16:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:17:05.621293 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f030f7-faaf-4aa6-bbca-825fe21ece7f None None] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:17:05.699104 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f030f7-faaf-4aa6-bbca-825fe21ece7f None None] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:17:05.705190 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:17:05.705999 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:05.706352 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:05.706691 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] PATH_INFO: `/v3/users/223b909a4d624d049c1c813ed391a3d3` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:05.708015 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=223b909a4d624d049c1c813ed391a3d3)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:17:05.710183 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:17:05.774759 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223b909a4d624d049c1c813ed391a3d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:05.791548 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223b909a4d624d049c1c813ed391a3d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:05.792329 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223b909a4d624d049c1c813ed391a3d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:05.792963 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223b909a4d624d049c1c813ed391a3d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:05.800500 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223b909a4d624d049c1c813ed391a3d3'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:05.819119 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1443/2905] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:17:05 2022] DELETE /identity/v3/users/223b909a4d624d049c1c813ed391a3d3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:17:05.823776 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028eb6-269c-49a2-9950-3b9f265bc300 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:05.824279 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028eb6-269c-49a2-9950-3b9f265bc300 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:05.824491 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f028eb6-269c-49a2-9950-3b9f265bc300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:05.850003 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-2f028eb6-269c-49a2-9950-3b9f265bc300 None None] MFA Rules not processed for user `2b6bbf928a744f868eafe13f4b7ff775`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:17:05.851301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-2f028eb6-269c-49a2-9950-3b9f265bc300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:05.873048 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=100 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:17:05.931507 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1463/2906] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 19:17:05.938984 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:17:05.940455 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:05.998912 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:17:06.016726 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d1b8c-5894-4b2c-8403-31f115d5c44e admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:17:06.021419 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:17:06.022277 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:06.022723 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:06.023324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:06.033662 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:17:06.035105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:17:06.046069 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1444/2907] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:17:06.318821 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c97ef-8b89-4a82-8755-2e2936b86a86 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:06.319343 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c97ef-8b89-4a82-8755-2e2936b86a86 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:06.319717 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c97ef-8b89-4a82-8755-2e2936b86a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:06.340992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-252c97ef-8b89-4a82-8755-2e2936b86a86 None None] MFA Rules not processed for user `4197287cdb4942f58a59e2016493fd84`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:17:06.342230 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-252c97ef-8b89-4a82-8755-2e2936b86a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:06.355982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:17:06.357450 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:06.382360 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc06838-4125-4b99-a431-c43bf593c956 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:17:06.388767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:17:06.389410 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:06.389767 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:06.390106 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] PATH_INFO: `/v3/projects/ab3143421f134832813afc985ac1386a` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:06.391366 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab3143421f134832813afc985ac1386a)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:17:06.393361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:17:06.413342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1464/2908] 10.222.0.94 () {58 vars in 958 bytes} [Mon Aug 29 19:17:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:17:06.448084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab3143421f134832813afc985ac1386a', 'request_id': 'req-5b33446c-2472-4607-aba1-948114d968f2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:06.454927 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab3143421f134832813afc985ac1386a', 'request_id': 'req-5b33446c-2472-4607-aba1-948114d968f2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:06.463121 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab3143421f134832813afc985ac1386a', 'request_id': 'req-5b33446c-2472-4607-aba1-948114d968f2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:06.475056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] The token cache is being invalidate because project ab3143421f134832813afc985ac1386a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:17:06.475471 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:06.476196 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:17:06.477884 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1445/2909] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:17:06 2022] DELETE /identity/v3/projects/ab3143421f134832813afc985ac1386a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:17:08.419589 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:08.420067 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:08.420172 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:08.443496 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:17:08.444841 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:08.507127 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1465/2910] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Aug 29 19:17:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:17:08.520108 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:17:08.521709 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:08.598249 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33446c-2472-4607-aba1-948114d968f2 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:17:08.604084 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:17:08.604758 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:08.605041 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:08.605348 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:08.613836 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:17:08.616327 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:17:08.625217 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1446/2911] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:17:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:17:18.499615 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=67 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:17:18.622339 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:17:37.562714 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:17:37.564516 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:37.624129 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2323cd0-1546-400c-bea4-6b8aecf65cf8 None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:17:37.627729 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:17:37.628341 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:17:37.628621 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:17:37.628895 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:17:37.631163 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:17:37.698411 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:17:37.699882 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:17:37.710428 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1466/2912] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:17:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:17:47.705252 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=50 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:04.095469 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:04.105679 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3df69a-6767-4c2d-8e5c-8d436a6c0f82 service swift] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:04.109308 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:04.109903 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:04.110184 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:04.110453 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:04.114889 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:04.234434 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:04.236020 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:04.259886 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1447/2913] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:05.562672 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:05.572866 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2c047-45b6-4a52-8761-5ec1c050d299 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:05.578333 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:05.579350 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:05.579838 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:05.580312 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:05.583184 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:05.605955 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:05.608500 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:05.627155 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1467/2914] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 19:18:14.256388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=43 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:15.619481 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=6 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:20.328764 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:20.342704 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e30894-4d25-43dc-80dc-69b53194f3e6 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:20.353923 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:20.358123 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:20.358982 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:20.360512 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:20.362941 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:20.382289 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:20.385456 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:20.402296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1448/2915] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:23.336301 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:23.338026 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:23.354241 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be086a99-0dd1-4010-b54b-f45654676916 service neutron] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:23.360203 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:23.360863 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:23.361132 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:23.361461 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:23.363362 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:23.376447 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:23.378332 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:23.390342 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1468/2916] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:26.238480 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:26.241046 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:26.252612 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec28e2-c5d9-4284-93d6-3ddc5ebdcfbc service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:26.256755 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70c6682b92694484b2fa2710314f547f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:26.257364 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:26.257643 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:26.257954 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:26.259735 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:26.334349 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:26.335820 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:26.345908 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1449/2917] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:32.865802 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:32.867430 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:32.930045 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:32.932105 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:32.947715 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c6fb1-842c-4688-8d4f-cc8be1ead895 service neutron] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:32.951309 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3779a97-df67-44b2-b6e1-1ab0f4c169af service nova] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:32.953056 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:32.953701 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:32.953991 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:32.954271 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] PATH_INFO: `/v3/users/8f1c0b2189a041fdb8dbe2a0290ffac8` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:32.955486 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1c0b2189a041fdb8dbe2a0290ffac8)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:32.957420 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:32.959229 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:32.960146 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:32.960953 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:32.961299 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:32.963267 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:32.976724 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:32.978258 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:32.989684 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1469/2918] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:18:33.012744 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1c0b2189a041fdb8dbe2a0290ffac8'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.026262 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1c0b2189a041fdb8dbe2a0290ffac8'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.027001 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1c0b2189a041fdb8dbe2a0290ffac8'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.027648 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1c0b2189a041fdb8dbe2a0290ffac8'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.035009 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1c0b2189a041fdb8dbe2a0290ffac8'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.049739 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1450/2919] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:18:32 2022] DELETE /identity/v3/users/8f1c0b2189a041fdb8dbe2a0290ffac8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 19:18:33.230399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:33.242647 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8802809-08e0-4d36-8712-8dd8873226c7 service placement] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:33.307553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:33.308262 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:33.308569 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:33.308899 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] PATH_INFO: `/v3/projects/6a4d940594f542808602c7260993f85f` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:33.310256 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4d940594f542808602c7260993f85f)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:33.312379 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:33.361754 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4d940594f542808602c7260993f85f', 'request_id': 'req-1817b001-93a0-458e-bda3-a29bd020c47a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.367962 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4d940594f542808602c7260993f85f', 'request_id': 'req-1817b001-93a0-458e-bda3-a29bd020c47a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.373655 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4d940594f542808602c7260993f85f', 'request_id': 'req-1817b001-93a0-458e-bda3-a29bd020c47a'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.386599 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] The token cache is being invalidate because project 6a4d940594f542808602c7260993f85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:18:33.386883 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.387605 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.389191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1470/2920] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:18:33 2022] DELETE /identity/v3/projects/6a4d940594f542808602c7260993f85f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:33.393309 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:33.394705 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:33.408909 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197113dd-2903-4871-87e7-07b03863a1af admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:33.457896 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:33.469301 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:33.470138 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:33.470427 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:33.470706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] PATH_INFO: `/v3/users/f181016e9d094ba98cf0cceac6ad5ff2` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:33.471937 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f181016e9d094ba98cf0cceac6ad5ff2)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:33.473942 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:33.526523 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f181016e9d094ba98cf0cceac6ad5ff2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.540112 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f181016e9d094ba98cf0cceac6ad5ff2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.540856 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f181016e9d094ba98cf0cceac6ad5ff2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.541470 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f181016e9d094ba98cf0cceac6ad5ff2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.548626 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f181016e9d094ba98cf0cceac6ad5ff2'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.565334 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1451/2921] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:18:33 2022] DELETE /identity/v3/users/f181016e9d094ba98cf0cceac6ad5ff2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:33.753553 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:33.755250 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:33.779997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817b001-93a0-458e-bda3-a29bd020c47a admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:33.786263 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:33.855445 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:33.856081 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:33.856367 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:33.856644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] PATH_INFO: `/v3/projects/35c86e18d47449a6bfb9a93397cec4e5` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:33.857763 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c86e18d47449a6bfb9a93397cec4e5)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:33.859766 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:33.908943 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c86e18d47449a6bfb9a93397cec4e5', 'request_id': 'req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.915592 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c86e18d47449a6bfb9a93397cec4e5', 'request_id': 'req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.921071 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c86e18d47449a6bfb9a93397cec4e5', 'request_id': 'req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.934400 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] The token cache is being invalidate because project 35c86e18d47449a6bfb9a93397cec4e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76733) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:18:33.934682 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.935329 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-47cdf808-41ef-43cd-9bac-a9f68e7ae7cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:18:33.936879 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1471/2922] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:18:33 2022] DELETE /identity/v3/projects/35c86e18d47449a6bfb9a93397cec4e5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:18:43.561569 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:43.931325 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:18:55.682567 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:18:55.684296 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:55.800543 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d15a9-62b2-4fda-a542-383f837c6010 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:18:55.888911 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8adead3860f144099efa11cefa83a37f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:18:55.889653 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:18:55.889938 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:18:55.890211 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:18:55.893353 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:18:55.913193 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:18:55.916107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:18:55.931492 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1452/2923] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:18:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:19:02.029618 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] REQUEST_METHOD: `POST` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:02.030269 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:02.030674 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:02.097306 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.auth.core [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] MFA Rules not processed for user `4ee3b5df27184223a916ceccb0ee4717`. Rule list: `[]` (Enabled: `True`). {{(pid=76733) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 19:19:02.098554 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:02.181051 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1472/2924] 10.222.0.94 () {58 vars in 967 bytes} [Mon Aug 29 19:19:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 19:19:02.223304 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:02.224989 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:02.323791 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa2a50-a9ae-46fe-8239-cdb769740ef2 service nova] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:02.330328 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0e57abdb8d4966a2066a85bbfbc15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:02.330955 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] REQUEST_METHOD: `GET` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:02.331252 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:02.331524 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:02.336862 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=100 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:02.340965 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:02.342943 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:02.352175 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1453/2925] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Aug 29 19:19:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 19:19:09.536680 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:09.538444 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:09.612206 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c0e8-c144-45a0-a6e9-8996086bbb5f None None] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:09.616807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:09.617440 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:09.617747 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:09.618021 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] PATH_INFO: `/v3/users/4197287cdb4942f58a59e2016493fd84` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:09.619195 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4197287cdb4942f58a59e2016493fd84)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:09.621176 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:09.670638 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4197287cdb4942f58a59e2016493fd84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:09.679041 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4197287cdb4942f58a59e2016493fd84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:09.679771 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4197287cdb4942f58a59e2016493fd84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:09.693271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4197287cdb4942f58a59e2016493fd84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:09.695399 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:09.702509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4197287cdb4942f58a59e2016493fd84'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:09.708842 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1473/2926] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:19:09 2022] DELETE /identity/v3/users/4197287cdb4942f58a59e2016493fd84 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:09.909135 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:09.922792 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee22216-3217-4c6a-a09c-0b760f512118 service swift] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:09.991646 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:09.992287 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:09.992573 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:09.992847 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] PATH_INFO: `/v3/projects/af3f2410b6e94037becf20e9c7908820` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:09.993973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3f2410b6e94037becf20e9c7908820)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:09.995973 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:10.046783 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3f2410b6e94037becf20e9c7908820', 'request_id': 'req-298da4f2-acc2-4839-aaa1-f894a4802e46'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.053157 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3f2410b6e94037becf20e9c7908820', 'request_id': 'req-298da4f2-acc2-4839-aaa1-f894a4802e46'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.062347 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3f2410b6e94037becf20e9c7908820', 'request_id': 'req-298da4f2-acc2-4839-aaa1-f894a4802e46'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.073388 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] The token cache is being invalidate because project af3f2410b6e94037becf20e9c7908820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:19:10.073668 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.074324 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.075958 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1454/2927] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:19:09 2022] DELETE /identity/v3/projects/af3f2410b6e94037becf20e9c7908820 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:10.080443 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:10.081964 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:10.095782 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656db8a-8cd9-4403-a069-231529d4a1d6 admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:10.159339 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:10.159996 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:10.160277 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:10.160550 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] PATH_INFO: `/v3/users/1a52c04256fa4e99b9d41d3836fa0581` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:10.161775 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a52c04256fa4e99b9d41d3836fa0581)` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:10.163809 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:10.220948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a52c04256fa4e99b9d41d3836fa0581'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.228968 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a52c04256fa4e99b9d41d3836fa0581'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.229644 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a52c04256fa4e99b9d41d3836fa0581'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.244316 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a52c04256fa4e99b9d41d3836fa0581'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.253894 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.notifications [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a52c04256fa4e99b9d41d3836fa0581'} {{(pid=76733) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.259992 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1474/2928] 10.222.0.94 () {58 vars in 1224 bytes} [Mon Aug 29 19:19:10 2022] DELETE /identity/v3/users/1a52c04256fa4e99b9d41d3836fa0581 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:10.699693 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Authenticating user token {{(pid=76732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:19:10.701696 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.fernet_utils [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:19:10.714907 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298da4f2-acc2-4839-aaa1-f894a4802e46 admin admin] Validating token access rules against request {{(pid=76732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:19:10.779192 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b6bbf928a744f868eafe13f4b7ff775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b2e035c0894ef7a5efbac9016e862c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:19:10.780116 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:19:10.780430 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:19:10.780706 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] PATH_INFO: `/v3/projects/a4d004040b8d4517a3b1f6b9f6acdf92` {{(pid=76732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:19:10.781926 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d004040b8d4517a3b1f6b9f6acdf92)` {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:19:10.783915 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] RBAC: Authorization granted {{(pid=76732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:19:10.817107 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:10.842279 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d004040b8d4517a3b1f6b9f6acdf92', 'request_id': 'req-87ef7a11-d082-4157-9e55-1a020d40d8cb'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.849253 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d004040b8d4517a3b1f6b9f6acdf92', 'request_id': 'req-87ef7a11-d082-4157-9e55-1a020d40d8cb'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.856320 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d004040b8d4517a3b1f6b9f6acdf92', 'request_id': 'req-87ef7a11-d082-4157-9e55-1a020d40d8cb'} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.867584 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] The token cache is being invalidate because project a4d004040b8d4517a3b1f6b9f6acdf92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=76732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 19:19:10.867876 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.868532 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG keystone.notifications [None req-87ef7a11-d082-4157-9e55-1a020d40d8cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=76732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 19:19:10.870044 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: [pid: 76732|app: 0|req: 1455/2929] 10.222.0.94 () {58 vars in 1233 bytes} [Mon Aug 29 19:19:10 2022] DELETE /identity/v3/projects/a4d004040b8d4517a3b1f6b9f6acdf92 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 19:19:20.256476 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:19:20.864361 dv-cinder-731261-6-iscsi devstack@keystone.service[76732]: DEBUG dbcounter [-] [76732] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=76732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 19:21:57.263245 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Authenticating user token {{(pid=76733) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 19:21:57.264757 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.337733 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2591-98f9-4279-85ba-109005da6cac admin admin] Validating token access rules against request {{(pid=76733) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 19:21:57.400106 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d28d8-e5a7-4767-bc3c-18430ffdcd7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '005b6400ae0c40bd8da5b9ddcc96770b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91f1720f91bc43f594467ca8b5fe0c35', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=76733) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 19:21:57.400718 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d28d8-e5a7-4767-bc3c-18430ffdcd7a service placement] REQUEST_METHOD: `GET` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 19:21:57.400997 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d28d8-e5a7-4767-bc3c-18430ffdcd7a service placement] SCRIPT_NAME: `/identity` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 19:21:57.401271 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d28d8-e5a7-4767-bc3c-18430ffdcd7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=76733) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 19:21:57.403807 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.fernet_utils [None req-ce8d28d8-e5a7-4767-bc3c-18430ffdcd7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=76733) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 19:21:57.415923 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d28d8-e5a7-4767-bc3c-18430ffdcd7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 19:21:57.417509 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d28d8-e5a7-4767-bc3c-18430ffdcd7a service placement] RBAC: Authorization granted {{(pid=76733) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 19:21:57.427948 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: [pid: 76733|app: 0|req: 1475/2930] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Aug 29 19:21:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 19:22:07.423191 dv-cinder-731261-6-iscsi devstack@keystone.service[76733]: DEBUG dbcounter [-] [76733] Writing DB stats keystone:SELECT=58 {{(pid=76733) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}